A Holistic Quality Evaluation, Selection and Improvement Approach driven by Multilevel Goals and Strategies

Organizations should establish business goals and check for their achievement in a systematic and disciplined way. In order to know if a business goal is achieved, it should be necessary to consider information need goals that also can require satisfying measurement and evaluation goals at operational level. Furthermore, if measurement and evaluation goals are not aligned with top-level business goals such as tactical or strategic level goals, the organization could waste its effort and resources. Usually, the different goals established in an organization are operationalized through projects. For a given project, strategies should be used in order to help in the goal achievement. A strategy defines a set of activities and methods to be followed for a specific goal purpose. Ultimately, to engineering all these issues in a systematic way, organizations should adopt a holistic evaluation approach supported by a set of integrated strategies. By means of a systematic literature review as research method, we have observed that very few approaches support integrated strategies and multilevel goals. To bridge this gap, we have developed a holistic quality multilevel and multipurpose evaluation approach that ties together multilevel goals, projects and integrated strategies. As contributions, this paper discusses an enhanced conceptual base (specified by ontologies) for linking business and information need goal concepts with project, strategy and nonfunctional requirements concepts. Then, it defines the step by step of our holistic quality evaluation approach, by listing the necessary activities to establish goals and projects at different organizational levels. Lastly, it specifies and illustrates evaluation scenarios for business/information need goal purposes such as understanding, improving, monitoring and controlling, comparing and selecting entities, which are supported by strategies and strategy patterns.


Introduction
In any modern business or organization, project objectives should not be established in an isolated manner.Usually, organizations establish business goals or objectives at strategic, tactical and operational levels [17].Basili et al. [4] indicate that a critical issue in an organization is the lack of linkage between goals formulated at a strategic level (or management level, considering their own words) with those objectives formulated at an operational (or project) level.Specifically, authors stated that "the problem is not that businesses fail to achieve their objectives, but rather that they do not always state these objectives explicitly or clearly enough to verify that they have indeed achieved those objectives.It is even less clear how a business translates its objectives into its lower organizational levels and into individual projects.At present, no methodology bridges the gap between business strategies and their projectlevel implementation".
Considering the first part of the above quote, two decades before Gilb et al. [14] had indicated, as the principle of fuzzy targets, that "projects without clear goals will not achieve their goals clearly".Considering the second part of the above quote, a decade before Park et al. [33] had said that the right establishment of business goals at different organizational levels determines much of the success in carrying out projects.As a consequence, if goals at an operational level are not in alignment with goals at tactical or strategic levels, the organization can direct its effort and resources in a wrong way.
In addition to business goals, it is also necessary to have valuable information which allows to know if the business goals were achieved.An information need goal is a support goal which is always related to a main or business goal [33].In our proposal, a particular kind of information need goal is the measurement and evaluation (ME) information need goal or objective, which is always driven by ME activities and methods.In summary, information need goals permit knowing the level of achievement of business goals as well as to give the necessary information to reach them.Information need goals are grounded in different and suitable types of analysis that can be made using qualitative and quantitative data and contextual information yielded by ME activities.It is important to remark that ME should not be an end in itself but a key factor to reach information need and business goals in an organization.As pointed out in [4], "quantitative data is a prerequisite to understanding the relationships between the business and project-level goals and verifying the achievement of objectives".
On the other hand, commonly, organizations arrange their work by means of projects in order to operationalize the established goals.An engineered way to organize the work is by performing project management.Project management allows planning, executing and controlling the activities and resources of the project regarding the adopted life cycle [34].For a given project life cycle, strategies should be used in order to help in the goal achievement.A strategy defines a specific course of action to be followed for a specific goal purpose [9,24].That is, it specifies concretely what should be done and how should be performed.Strategy is a frequently used and broad term, so for our purposes, we have defined it as: "principles, patterns, and particular domain concepts and framework that may be specified by a set of concrete processes, in addition to a set of appropriate methods and tools, as core resources, for helping to achieve a project's goal purpose" [5].
Regarding ME strategies, we conducted a systematic literature review [31], in which we have observed that very few approaches supported integrated strategies.Also, the research allowed us to detect that just one integrated strategy (GQM + Strategies [4]) included multilevel goals.The premise for an integrated strategy is that it supports three integrated capabilities simultaneously, namely: (1) the domain conceptual base and framework, (2) the process perspective specifications, and (3) the method specifications.In [31], another integrated strategy was GOCAME (Goal-Oriented Context-Aware Measurement and Evaluation) [26].For this strategy, the first capability, is named the C-INCAMI (Contextual-Information Need, Concept Model, Attribute, Metric and Indicator) conceptual framework, which explicitly specifies the ME terms, properties, relationships and constraints, in addition to their grouping into components.The second capability is the process specifications [5], which describes a set of activities, tasks, inputs and outputs, artifacts, roles, and so forth.Besides, process specifications can consider different process perspectives such as functional, behavioral, informational and organizational [11].Usually, process specifications primarily state what to do rather than indicate the particular methods and tools (resources) used by specific activity descriptions.The third capability provides the ability to specify methods (such as metric and indicator specification templates), which ultimately represent the particular ways to perform the ME tasks.
In addition to the GOCAME strategy, we have defined a set of strategy patterns [35,38] for the measurement, evaluation and change (MEC) domain.A strategy pattern represents a reusable solution for instantiating the suitable strategy considering the project's goal purpose, and the amount of included quality views.(Note that the quality view concept [36,38] stems from the relationship between an evaluated entity supercategory -e.g., resource, process, software product, system, among others-and its quality focus -e.g., resource quality, process quality, internal quality, external quality, among others.In turn, for each quality focus a tailored quality model which includes characteristics and attributes should be used for evaluation, selection or improvement purposes).
Regarding the above mentioned research ( [4,33,34]) and raised issues such as "...At present, no methodology bridges the gap between business strategies and their project-level implementation" [4], we also consider that it is paramount for software organizations to have a systematic and holistic approach which is able to establish and align goals at different organizational levels, and arrange work by means of ME/MEC projects and strategies for helping to achieve these goals.In this direction, Basili et al. have developed GQM + Strategies.However, we observe some opportunities for improvement to this approach, as we highlight in Section 2. As a result, we have developed a systematic approach that considers and relates multilevel goals, projects, strategies and strategy patterns, which is called the Holistic Quality Multilevel and Multipurpose Evaluation Approach (hereafter, for short: Quality Evaluation Approach).This approach embraces four key aspects or principles: i) the definition of multilevel business and information need goals; ii) the definition of different evaluation purposes for goals that also include quality views and their relationships as well; iii) the formulation of ME/MEC projects for operationalizing goals; and iv) the adoption of strategy patterns for the instantiation of specific strategies that help to achieve goal purposes.
These four principles of the Quality Evaluation Approach rely also on conceptual bases which are structured into ontologies.For instance, the abovementioned C-INCAMI conceptual framework deals with the ME domain and represents components such as nonfunctional requirements, measurement, evaluation and analysis.More recently, we have developed the quality multiview modeling framework that includes an ontology of quality views [36].A year ago, we added the goal and project components [37] for linking business and information need goal concepts with project, strategy and nonfunctional requirements concepts.Therefore, the main contributions of this research are summarized in the following list: i) The specification of a couple of new terms included into the goal and project conceptual components -which were left implicit in [37]-such as types of organizational levels and strategy pattern.As a result, the enhanced conceptual base is able to relate business and information need goal terms with ME information need, entity category, quality focus, and quality view terms in addition to with project, strategy and strategy pattern concepts.
ii) The definition of the Quality Evaluation Approach stepwise, which lists the necessary activities to establish goals and projects at different organizational levels.The step-by-step applicability can help stakeholders in the process of relating business goals with ME information need goals in addition to formulate and perform ME/MEC projects by using the suitable strategies.Also, while exemplifying the step-by-step sequence, an evaluation scenario for improving the Facebook mobile app is used.
iii) The documentation of evaluation scenarios for business/information need goals at operational level supported by strategies and strategy patterns.For this end, we specify a template which contains a set of items such as the business goal purpose, the evaluation strategy pattern in conjunction with its process specification, among others.We have identified different evaluation strategies that realize purposes aimed at understanding, improving, monitoring and controlling, comparing and selecting entities regarding also the amount of quality views.Examples of scenarios in which each strategy helps to achieve a given purpose are documented as well.
Following to this Introduction, Section 2 motivates our proposal and analyzes related work on ME/MEC approaches and integrated strategies that consider multipurpose and multilevel goals, which in turn are supported by conceptual bases.Section 3 deals with the goal and project conceptual components and their relations with the previously developed C-INCAMI conceptual components.Section 4 defines the step by step of our Quality Evaluation Approach and illustrates its applicability through a proof of concept using as scenario the improvement of the Facebook mobile app.Section 5 documents evaluation scenarios for business/information need goals at operational level for any organization, by addressing well-established purposes supported by strategies and strategy patterns.Finally, Section 6 summarizes the main contributions of this research and outlines future work.

Motivation and Related Work
In Software Engineering there exists research that addresses the importance of linking and aligning measurement goals usually formulated at an operational level of an organization, with business goals formulated at this level or often at higher levels such as tactical or strategic [2,4,15,33,40].This alignment is paramount for sound and robust decision-making endeavors since ME information need goals can give meaningful information for learning and knowing in what extent a business goal has been achieved.In some cases, ME information need goals can also give information on how to reach the business goal.
To engineer these issues, it is ultimately paramount for software organizations to have a holistic and systematic evaluation approach.This approach should be able to establish and align goals at different organizational levels, and arrange work by means of ME/MEC projects and strategies for helping to achieve business goals, information need goals, and ME information need goals for different purposes such as to understand, monitor and control, improve, compare and select, among others.In any case, by running a specific ME/MEC strategy for a given purpose produces data (by using metrics) and information (by using indicators) which are input for analyzing results, and ultimately, for decision making.
As indicated in the Introduction Section, such a holistic approach should consider core aspects such as the definition of multilevel business and information need goals; the definition of different purposes for evaluation goals that also include quality views and their relationships; the formulation of ME/MEC projects for operationalizing goals; and the instantiation of specific strategies that help to achieve goal purposes.(Note that ME/MEC strategies can stem from the instantiation of strategy patterns).Additionally, as the reader can surmise, given the abundant multidomain terminology that the holistic approach embraces, a minimum set of terms, attributes and relationships should be explicitly specified and agreed.
However, reviewing the related literature, we have not detected approaches with robust conceptual bases (i.e., ontologies) that integrate terms such as business goal, organizational level, information need and ME information need goal, strategy, strategy pattern, project, ME/MEC project, among others, except in Barcellos et al. [2] who include some of these terms.On the other hand, considering that a strategy is an important resource in helping to achieve a project's goal purpose, it should be noted, as discussed in [31] that there exist few documented strategies that integrate simultaneously the process specifications, the method specifications, and the ME conceptual base capabilities.As commented previously, we conducted a systematic literature review as research method, in which the main aim was the selection of integrated strategies for evaluation and comparison purposes.Basically, among the preselected strategies were GQM + Strategies and GOCAME, among others.It is important to remark the research allowed us to detect that only GQM + Strategies supports multilevel goals.Furthermore, there are different ME purposes documented elsewhere [6,8,12,18,20,34] such as to understand, improve, monitor and control, compare and select, among others.Consequently, different strategies should be considered for each purpose, varying slightly each other in its own process and method specifications, as we discuss and illustrate in Section 5. Therefore, having well-specified processes and methods for each strategy can foster their applicability and communicability.These issues have often been neglected in the literature as well.
Regarding the ME approaches, GQM + Strategies is an integrated strategy.It includes a goal-oriented framework for the design and implementation of software measurement projects at different organizational levels.Unlike its predecessor, GQM (Goal Question Metric) [3], the business goals that GQM + Strategies defines can be aligned at different organizational levels through the establishment of strategies.In [4], strategies define objectives for reaching goals and require the definition and fulfillment of lower level goals.Therefore, business goals are linked to measurement goals using GQM.GQM + Strategies has a terminological base structured as a glossary where the main used terms are defined.But it lacks the semantic richness that an ontology provides.Also, well-established process specifications for different strategies regarding different goal purposes are missing.
Another measurement approach widely accepted in the industry that helps manage software development projects is PSM (Practical Software Measurement) [23].It is an information-oriented approach that describes a software measurement process, also being part of a comprehensive management program and software development project management.PSM describes how to define and integrate measurement requirements, collect and analyze measurement data and implement the entire measurement process in an organization.PSM was one of the sources for the development of the ISO/IEC 15939 standard [19], as indicated in http://www.psmsc.com/iso.asp.And after the formal appearance of this ISO document, PSM was updated according to this standard as well.We consider PSM an integrated strategy since includes process specifications, method specifications, and a glossary of terms for the ME domain.However, the explicit support to multilevel goals and multipurposes is missing.That is, PSM (likewise GQM + Strategies) considers just one strategy for all the purposes.
Another related work is the Goal-Driven Measurement approach [33], which describes a process for the definition of measurement goals aimed at helping to understand aspects of the organizational goals.The process begins indicating that the organization should establish business goals at any organizational level.From these goals, questions or issues related to what stakeholders want to know or learn emerge, as also pointed out in [39].These issues allow identifying quantitative information through the decomposition of the business goal into related subgoals.With the list of subgoals and issues, entities and attributes are identified, following the GQM model and templates.This approach offers guidelines that serve as an important reference for engineers and practitioners, since these detail the approach process through the goal decomposition to the measure quantification and analysis.It also uses the terms that GQM defines.But authors in [33], neither explicitly define concepts such as business goal, organizational level, information need goal, strategy and strategy pattern, among others, nor specify different strategies regarding the abovementioned ME purposes.
Additionally, Goethert and Fisher [15] describe the GQ(I)M (Goal-Question-Indicator-Measurement) approach that combines the most prominent aspect of the strategy described in [33] with the Balanced Scorecard [21] paradigm for the decomposition of strategic goals into subgoals.GQ(I)M approach is used to systematically establish organizational goals for each quadrant that Balanced Scorecard defines.Also, it helps identifying and defining measures and indicators.GQ(I)M includes a glossary with the definition of some terms, which are not present in GQM.It is worthy to remark that GQ(I)M does not deal with the use of integrated strategies to fulfill business goals from information need goals.
In [2], Barcellos et al. define a measurement goal subontology.They argue that the measurement should be aligned with organizational goals in order to produce useful data for decision making.This contains terms, relationships and restrictions related to measurement, goal and organization concepts.Also authors state that indicators are the measures which can be used to assess the level of goal achievement.However, the use of integrated strategies are not described in their research.Unlike [2], our proposal formally establishes the use of integrated strategies for helping to achieve both business and information need goals for different evaluation purposes.
The business goal subontology (specified in the next section) discusses some concepts which are not modeled in [37].For example, the strategy pattern concept which represents a knowledge asset.A ME/MEC strategy pattern embeds a reusable and customizable solution for a recurrent ME/MEC project problem in similar contexts [38].Likewise the process guideline offered in [33], we describe in Section 4 the step-by-step applicability of our approach, which in [37] was not explicitly considered.Lastly, a contribution of this work, which was left apart in [37], is the documentation of evaluation scenarios for business/information need goals at operational level supported by strategies and strategy patterns.Hence, in Section 5, we specify a template which contains a set of items such as the business goal purpose, the evaluation strategy pattern in conjunction with its general process specification.We have identified different evaluation strategies that deal with purposes aimed at understanding, improving, monitoring and controlling, comparing and selecting entities regarding also the amount of quality views.

Linking Business and Information Need Goals with Project, Strategy and Strategy Pattern: A Conceptual Base Description
As commented above, the proposed Quality Evaluation Approach is grounded on robust conceptual bases as a way to formally define the terms and relationships involved in quality measurement, evaluation and improvement issues.There exist different ways to structure conceptual bases such as glossaries, taxonomies and ontologies.In the Quality Evaluation Approach, conceptual bases are structured as ontologies.The benefits of ontologies are well known.Just to write down an often cited quote: "An ontology may take a variety of forms, but necessarily it will include a vocabulary of terms, and some specification of their meaning.This includes definitions and an indication of how concepts are inter-related which collectively impose a structure on the domain and constrain the possible interpretations of terms.An ontology is virtually always the manifestation of a shared understanding of a domain that is agreed between a number of agents.Such agreement facilitates accurate and effective communication of meaning, which in turn leads to other benefits such as inter-operability, reuse, and sharing" [41].Therefore, we can state that an ontology is a richer mechanism than others for structuring a conceptual base, which can also be specified by means of ontological languages such as OWL (Ontology Web Language).Hence, ontological languages are able to support semantic processability.
Originally, the C-INCAMI conceptual framework was composed of six conceptual components such as nonfunctional requirements, measurement, evaluation, context, quality view and project (see these packages in Fig. 1).The terms, attributes and relationships of these components arose from the ME ontology documented in [25,26] and from the quality view ontology formalized in [36,38].In a complementary research published in [5], the measurement and evaluation components were semantically enriched by the process ontology.Recently, in [37], we have argued that C-INCAMI had not the necessary terms for linking ME information need goals with business goals.This is important for representing the alignment between goals at different organizational levels, such as strategic, tactical and operational.Hence, we have added the business goal component and enlarged the project component as shown in Fig. 1.
Basically, these components specify both business and information need goals at different organizational levels, which can be operationalized by projects and achieved by means of strategies.The terms, attributes and relationships of these conceptual bases were defined considering documents such as [1,2,4,10,16,19,30,34,40] with the aim of having some adherence or contrast to well-known sources.However, in [37] a couple of terms were left implicit, e.g., the specific levels of the Organizational Level term and the Strategy Pattern term and its attributes.Therefore in Fig. 1, we add these terms for the business goal and project components in addition to the previous ones included in [37].In tables 1, 2 and 3 the definition of all terms, relationships and attributes are presented.In the sequel, descriptions of these terms are made.Note that terms are highlighted next in italic the first time they appear in the text.
An organization is an entity that comprises people and is structured and managed to establish and pursue goals.The organization establishes goals which contain an explicit declaration (statement) about the major purpose that should be achieved in a period of time (timeframe).The purpose of an organizational goal is the rationale for achieving it (e.g., to develop, maintain, increase, reduce, understand, monitor, control, improve, predict, analyze, among others).The established time frame for the achievement of a goal can range from short and intermediate term to long term.Also, goals can be classified into business and information need goals.Business goals are the main or primary goals that an organization sets considering its mission and vision.Goals can be formulated at different organizational levels.An organizational level represents a management and decision-making level.Commonly, three levels are identified in the literature [4,17,21] such as strategic, tactical and operational levels.In turn, a business goal can be divided into business subgoals.
On the other hand, information need goals are support goals for business goals.Usually, they provide useful information in order to know the degree of achievement of business goals.An information need goal can also require ME information need goals.The latter is a more specific type of information need goal which is driven by ME activities.Note that a ME information need specifies an object to be evaluated (entity category) considering also a quality focus (see in Fig. 1 the nonfunctional requirements component, which includes the quality view one).It is a main or primary Goal that the Organization intends to achieve.

Change Project
It is a Project for operationalizing a Business Goal with the purpose of changing the current state of an entity.Note: Different kinds of changes (e.g., adaptive, perfective, corrective changes) can be made in maintenance projects.Also, changes can be made on entities in development projects.

Development Project
It is a Project for operationalizing a main Business Goal with the purpose of building a new product or system.

Domain Conceptual Base
It is a terminological base in which, for a given domain, the main terms or concepts are explicitly defined.Note: A Conceptual Base can be structured for instance in a glossary, taxonomy, or ontology, amongst other ways.

Goal (synonyms Organizational Goal, Objective)
The statement of the aim to be achieved by the Organization which considers the propositional content of a purpose in a given time frame.Note: In some literature such as [30], a goal compared to an objective tends to be general rather than specific, longer term rather than shorter term.

Information Need Goal
It is a Goal intended to get insight for a given Business Goal.Note: Information Need, as per [19], is defined as "Insight necessary to manage objectives, goals, risks, and problems".

MEC Project
It is a Project for operationalizing a Business Goal with the purpose of improving an entity by performing ME-driven changes.Note: A MEC Project comprises both Change and ME Projects.ME Information Need It is an Information Need Goal driven by measurement and evaluation activities.

ME Project
It is a Project for operationalizing a Business Goal with the purpose of understanding or monitoring the current state of an entity in addition to selecting alternatives among entities.

Method Specification
It is the representation of a method.Note 1: Method is the specific and particular way to perform the specified steps in the description of a Work Definition.Note 2: The specific and particular way of a method -i.e., how the described steps in a work definition should be made-is represented by a procedure and rules [5].

Organization
It is an entity comprising people that is structured and managed to establish and pursue organizational Goals and is affected by and affects to its environment or context.Note: Most organizations have a level-oriented management structure that determines relationships between the different members and the activities, and subdivides and assigns roles and authority to perform different tasks.

Organizational Level
It represents a management and decision-making level in which Organization's Business Goals are formulated and Information Need Goals are taken into account.Note: Usually, long-term Business Goals are formulated at strategic Organizational Level, while short-term Business Goals are formulated at operational Organizational Level.

Process Specification
It is a model which relates a set of process elements such as activities, tasks, inputs and outputs, pre-and post-conditions, artifacts, roles, amongst others.Note 1: A Process Specification can consider different process perspectives such as functional, behavioral, informational and organizational [11].Note 2: Usually, Process Specifications primarily state what to do rather than to represent how to do activity descriptions.

Project
It is an entity representing a temporary and goal-oriented endeavor with definite start and finish dates, which considers a managed set of interrelated activities, tasks and Resources aimed at producing and modifying unique work products (i.e., artifacts, services or results) for satisfying a given requester need.

Project Life Cycle
The series of phases that a Project passes through from its initiation to its closure [34].
Note: Examples of phases to be managed in any Project are planning, scheduling, monitoring, among others.

Project Management
It is the set of managerial processes and activities intended to achieve the Goal operationalized by a Project.Note: The application of knowledge, skills, tools, and techniques to project activities to meet the project requirements, as per [34].

Project Management Plan
The document that describes how the Project will be executed, monitored, and controlled [34].

Resource
Asset assigned to a Work Definition.Note: An asset is an entity (e.g., agent, people, Strategy, method, tool, etc.) with added value for an Organization [5].

Strategy
Principles, patterns, and particular domain concepts and framework that can be specified by a set of tailored processes, in addition to a set of appropriated methods and tools, as core Resources, for helping to achieve the Project's Goal purpose.

Strategy Pattern
It is a knowledge asset that includes a reusable and customizable solution to a recurrent Project problem in similar situations.Note: A MEC strategy pattern is a reusable and customizable solution which deals with MEC project problems.

(Strategy Pattern) Structure
Generic and instantiable solution that the strategy pattern offers.Note: The structure of a strategy pattern aggregates three capabilities simultaneously, namely: (i) a Domain Conceptual Base; (ii) Process Specifications, which embrace process perspectives; and, (iii) Method Specifications [31].

Work Definition
Abstract entity which describes the work by means of consumed and produced work products, conditions and involved roles.Note: Work represents a process, an activity or a task [5].An organization arranges work by means of projects.In turn, projects operationalize established organizational goals.There exist different types of projects such as development, maintenance, evaluation, among others.For example, a development project operationalizes a business goal that has as purpose to develop a new software product or system.Particularly, in this work we focus on ME projects, which operationalize ME information need goals, as well as on MEC projects which operationalize both business goals and their related ME information need goals with the purpose of improvement.Fig. 1 shows that a MEC project is composed by a ME subproject and a change subproject in which changes are driven by measurement and evaluation.A change project operationalizes a business goal with the purpose of changing or improving the current state of an entity.
Additionally, project management is the set of managerial processes aimed to achieve the goal operationalized by a project.In [34], it is defined as "the application of knowledge, skills, tools, and techniques to project activities to meet the project requirements".The project management process produces a project management plan, that is, the document which describes how the project should be executed, monitored and controlled.Besides, a project adopts a project life cycle which indicates the stages the project goes through from its beginning to its end.The project life cycle involves at least resources and work definitions and uses strategies.We have defined the strategy term in the Introduction Section, so we consider that definition in Table 1.Hence, a strategy is a resource which helps to achieve a goal.
It is worthy to remark that our main line of research was devoted to ME and MEC strategies.Therefore, our developed strategies are intended to help to reach goals that are operationalized by ME or MEC projects.In addition to strategies, we have built recently a set of strategy patterns [32,35,38].A strategy pattern is a knowledge asset that includes a reusable and customizable solution to a recurrent project problem in similar situations.Particularly, a ME/MEC strategy pattern is a reusable and customizable solution which deals with ME/MEC project problems.Fig. 1 shows that a strategy may instantiate a strategy pattern.In turn, a strategy pattern has a structure compound of three integrated capabilities, namely: the domain conceptual base, process specifications, and method specifications.The domain conceptual base embraces a terminological base for a given domain, for instance, the ME domain.The second capability describes what to do by means of a model which relates a set of process elements such as activities, tasks, inputs, outputs, pre-and post-conditions, artifacts and roles.A process specification can also consider different process perspectives [5,11].The third capability represents how an activity should be carried out using a method specification based on a procedure and rules.A specific strategy pattern is documented in Appendix I, which includes the following items: name, alias, intent, motivation, applicability, structure, known uses, and scenario of use.
Finally, note that the terms included in the business goal component are the minimum and necessary ones for describing goals.The same occurs with the project component which relates terms as project, strategy and strategy pattern.The reader can surmise that, for instance, for the project management term there could be more specific related terms, but are not represented in our subontology due to its intended scope and objective.
In the next section, we instantiate these key terms for a MEC project, aimed at illustrating the main activities of the proposed Quality Evaluation Approach.

Quality Evaluation Approach: Step-by-step Applicability
As commented in the Introduction Section, we consider software organizations should foster a systematic approach which is able to establish and align goals at different organizational levels, and arrange work by means of projects and strategies for helping to reach these goals.In this direction, we have developed the Holistic Quality Multilevel and Multipurpose Evaluation Approach.This approach is based on four principles, viz.i) the definition of multilevel business and information need goals; ii) the definition of different evaluation purposes for goals that also include quality views and their relationships as well; iii) the formulation of ME/MEC projects for operationalizing goals; and iv) the adoption of strategy patterns for the instantiation of specific strategies that help to achieve goal purposes.It also relies on conceptual bases (subontologies) related to these aspects as analyzed in Section 3.
In this Section, we present the step by step of our approach which defines the necessary activities to establish goals and projects at different organizational levels.The step-by-step applicability can help stakeholders in the process of relating business goals with ME information need goals in addition to formulate and perform ME/MEC projects.Next, we list the approach main steps (S) or activities: In order to illustrate the approach steps, we employ a proof of concept for the evaluation and improvement of the Facebook mobile app considering multilevel goals, a specific strategy and its corresponding strategy pattern.This particular Facebook evaluation scenario is shown in Fig. 2

using BPMN (Business Process Model & Notation).
Additionally, many terms of the analyzed components are instantiated in Fig. 3 and 4 following the same scenario.Let's suppose "Facebook Inc." organization establishes at strategic level (S.1 in Fig. 2) the following business goal "Increase 20% the number of the Facebook mobile app' users for the 2016 year", which is operationalized by a specific project (see also Fig. 3).For this business goal, two business subgoals are established in S.2: (a) "Increase 5% the Facebook mobileapp advertising" (at tactical level), and; (b) "Improve 10% the Facebook mobileapp usability in 6 months" (at operational level).
With regard to the (b) business subgoal, it can be achieved by making changes on the Facebook mobile app, which are driven by ME activities.Therefore, this subgoal is operationalized by a MEC project (see Fig. 3).To give supporting information to this business subgoal an information need goal can be established in S. 3. The statement of this information need goal is "Analyze if usability has improved 10% after changes" across the 6-month time frame.Therefore this information need subgoal will allow to know the extent to which the (b) business subgoal has been achieved after making MEC activities.(Notice that the (a) business subgoal here is not analyzed since surely it will require subgoals, projects and strategies related to the marketing area, which is outside the scope of this article.Consequently, its respective step for the information need goal is not modeled in Fig. 2).
Going a step forward (S.4.1), a strategy pattern may be selected for the (b) subgoal.To this end, the business goal statement ("Improve 10% the Facebook mobileapp usability in 6 months"), which embeds the "improve" purpose is compared against the intent field in the template of each strategy pattern stored in the catalog.After performing this matching -which also considers the amount of quality views, as we see later on-the selection of the suitable strategy pattern is made.
Specifically, the "Improve 10% the Facebook mobileapp usability..." statement in the (b) business subgoal involves one quality view, i.e., the "System Quality View".This is so, because the stated concrete entity is the "Facebook mobile app" that belongs to the " to the "External Quality" focus.Therefore, GoMEC_1QV ( One Quality View) [35,38] is the suitable strategy pattern to be selected.This pattern is applicable to MEC projects in which the purpose is to improve the quality focus of the evaluated entity for one quality view GoMEC_1QV is documented in Appendix I Once S.4.1 was performed, the S.4.2 step considers the ME information need/business goals that the pattern process specification determines.Basically, GoMEC_1QV establishes three subgoals viz.(i) unders quality state of the entity, (ii) make changes on it, and (iii) understand the ulterior quality state (the improvement) after changes.This situation is also analyze Particularly, in Fig. 3, the three concrete ME inform mobile app usability weaknesses", (ii) " "Understand the Facebook mobile app usability after changes subgoal is a business goal, which is operationalized by a change project, while the rest of the subgoals are ME information need goals, which are operationalized by ME projects.In consequence, the three subprojects compose the MEC project which is carried out by using a particular strategy.
In the S.  " that belongs to the "System" entity category, and the "Usability" characteristic that is related " focus.Therefore, GoMEC_1QV (Goal-oriented Measurement, Evaluation and Change for ] is the suitable strategy pattern to be selected.This pattern is applicable to MEC projects in which the purpose is to improve the quality focus of the evaluated entity for one quality view GoMEC_1QV is documented in Appendix I. Once S.4.1 was performed, the S.4.2 step considers the ME information need/business goals that the pattern process specification determines.Basically, GoMEC_1QV establishes three subgoals viz.(i) unders quality state of the entity, (ii) make changes on it, and (iii) understand the ulterior quality state (the improvement) analyzed in subsection 5.2.Particularly, in Fig. 3, the three concrete ME information need/business goals are: (i) "Understand the Facebook ", (ii) "Apply changes on the Facebook mobile app current version Understand the Facebook mobile app usability after changes".Analyzing this figure, we also see that the (ii) subgoal is a business goal, which is operationalized by a change project, while the rest of the subgoals are ME information need goals, which are operationalized by ME projects.In consequence, the three subprojects compose the project which is carried out by using a particular strategy.In the S. " characteristic that is related Measurement, Evaluation and Change for ] is the suitable strategy pattern to be selected.This pattern is applicable to MEC projects in which the purpose is to improve the quality focus of the evaluated entity for one quality view.Note that Once S.4.1 was performed, the S.4.2 step considers the ME information need/business goals that the pattern process specification determines.Basically, GoMEC_1QV establishes three subgoals viz.(i) understand the current quality state of the entity, (ii) make changes on it, and (iii) understand the ulterior quality state (the improvement) Understand the Facebook Apply changes on the Facebook mobile app current version", and (iii) also see that the (ii) subgoal is a business goal, which is operationalized by a change project, while the rest of the subgoals are ME information need goals, which are operationalized by ME projects.In consequence, the three subprojects compose the

Aware Measurement, Evaluation and Change)
zes the process and method Scenario instantiation where a Business Goal from the strategic level is decomposed in subgoals which are Goals and ME Information Need Goals.Note: FmApp means Facebook mobile In order to illustrate the above (i) ME information need goal, Fig. 4 instantiates some terms of the and nonfunctional requirements components, which were specif mobile app usability weaknesses" subgoal, the quality focus is " The External Quality focus is represented by a quality (Calculable Concept), as well as the " subcharacteristics combine attributes.Note that Fig. 4 includes just one attribute (i.e., the " controls" attribute) in order not to clutter the diagram.The complete requirements tree specification for this case study can be found in [29].Furthermore, the above ME information need also specifies the object to be evaluated, which is the "Social network application entity super category.Finally, the association between the quality focus and the entity super category determines the quality view.For our proof of concept, we instantiated the " Once the ME/MEC projects, for those goals that require ME activities were planned and scheduled resources such as the suitable strategy-, the S.5 step is performed.
Lastly, in the S.6 step, evaluators should check the achievement of business goals by analyzing information need goals.In our approach, this is a bottom operational level allows to inform not only that level, but also higher levels as the information is aggregated and rolled up.
To our scenario, using the measure and indicator values that the MEC project yielded can "Analyze if usability has improved 10% after i.e., "Improve 10% the Facebook mobile app usability in 6 months not achieved and there would be time within the 6 performed using GOCAMEC.
Ultimately, because all multilevel goals are to some extent linked, measurement, evaluation and improvement planning and results are organization-wide rather than limited to a single project or depar in [4].

Evaluation Scenarios for Business and Information Need Goals at Operational Level supported by Strategies and Strategy Patterns
As described above, the Quality Evaluation business goal at top level should be refined into operational level.Also, it promotes the formulation of ME/MEC projects using strategy patterns accordingly.two main issues should be identified from the amount of quality views to be considered different purposes are documented [6,7,10,12,13, 18, 20, 3 understand, characterize, improve, predict, Looking at the above literature, it is worthy to remark that there is no broad consensus about ms for the quality_view and nonfunctional requirements components for the given ME Note: FmApp means Facebook mobile Application and ME, Measurement and Evaluation In order to illustrate the above (i) ME information need goal, Fig. 4 instantiates some terms of the components, which were specified in Fig. 1.For the "Understand the Facebook " subgoal, the quality focus is "External Quality".The External Quality focus is represented by a quality concept model which includes the "Usability (Calculable Concept), as well as the "Understandability" and "Operability" subcharacteristics.In turn, subcharacteristics combine attributes.Note that Fig. 4 includes just one attribute (i.e., the "Permanence of main to clutter the diagram.The complete requirements tree specification for this case Furthermore, the above ME information need also specifies the object to be evaluated, Social network application".This is an instance of the entity category which pertains to the " entity super category.Finally, the association between the quality focus and the entity super category determines the quality view.For our proof of concept, we instantiated the "System Quality View".
Once the ME/MEC projects, for those goals that require ME activities were planned and scheduled , the S.5 step is performed.Lastly, in the S.6 step, evaluators should check the achievement of business goals by analyzing information need goals.In our approach, this is a bottom-up analysis in which the interpretation of the information need goal at form not only that level, but also higher levels as the information is aggregated and To our scenario, using the measure and indicator values that the MEC project yielded in the ME activities nalyze if usability has improved 10% after changes" and then to understand if the linked business subgoal, mprove 10% the Facebook mobile app usability in 6 months" has been achieved.If the case were that it was not achieved and there would be time within the 6-month time frame, a new change and evaluation cycle can be Ultimately, because all multilevel goals are to some extent linked, measurement, evaluation and improvement wide rather than limited to a single project or department, as Basili

Business and Information Need Goals at Operational Level supported by Strategies and Strategy Patterns
Quality Evaluation Approach promotes the decomposition of main business goal should be refined into operationalizable business goals at lower level .Also, it promotes the establishment of information need goals for business goal using strategy patterns accordingly.In order to select the suitable strategy pattern, two main issues should be identified from the statement of an operational business goal, viz.o be considered.With regard to goal purposes in the software measurement [6,7,10,12,13,18,20,34].These evaluation purposes are predict, assess, select alternatives, change, monitor and control Looking at the above literature, it is worthy to remark that there is no broad consensus about In order to illustrate the above (i) ME information need goal, Fig. 4 instantiates some terms of the quality_view Understand the Facebook Usability" characteristic " subcharacteristics.In turn, the Permanence of main to clutter the diagram.The complete requirements tree specification for this case Furthermore, the above ME information need also specifies the object to be evaluated, ance of the entity category which pertains to the "System" entity super category.Finally, the association between the quality focus and the entity super category determines the Once the ME/MEC projects, for those goals that require ME activities were planned and scheduled -by assigning Lastly, in the S.6 step, evaluators should check the achievement of business goals by analyzing information need up analysis in which the interpretation of the information need goal at form not only that level, but also higher levels as the information is aggregated and in the ME activities, we " and then to understand if the linked business subgoal, " has been achieved.If the case were that it was nge and evaluation cycle can be Ultimately, because all multilevel goals are to some extent linked, measurement, evaluation and improvement tment, as Basili et al. state

Business and Information Need Goals at Operational Level
business goals.That is, a level, specifically at information need goals for business goals, and the In order to select the suitable strategy pattern, the purpose and the easurement literature, purposes are basically to control.Looking at the above literature, it is worthy to remark that there is no broad consensus about the meaning of purposes.Most of these works consider that characterize and understand are the same purpose, which involve understanding or forming a snapshot of the current state of an entity for establishing baselines for future assessment [7,13,18].Also, the improve purpose is related with the identification of risk, root causes, inefficiencies and other opportunities for improving the entity quality.Sometimes, the improve purpose also implies introducing changes [18].
On the other hand, the monitor purpose aims at following the trends or evolution of the performance or state of an entity [6].Monitoring is defined in [20] as "continual checking, supervising, critically observing or determining the status in order to identify change from the performance level required or expected".Often, related to the monitor purpose is the control purpose.The latter is devoted to identifying deviations that influence the state or performance of processes and products in order to alleviate risks [6].
Additionally, control is defined in [34] as "comparing actual performance with planned performance, analyzing variances, assessing trends to effect process improvements, evaluating possible alternatives, and recommending appropriate corrective action as needed".In some cases, monitoring and controlling are considered as a single purpose [6,18], which involves a continual evaluation and sometimes introducing changes in order to meet the expected quality level.
In summary, we observe an opportunity to gain consensus on the meaning of purposes.To this aim, an ontology for evaluation purposes should be developed.This section is not devoted to develop such an ontology but rather, it is focused on identifying a set of evaluation scenarios or situations related to evaluation purposes commonly used in the literature.
Therefore, we will specify scenarios for understanding, improving, monitoring and controlling, comparing and selecting purposes.Specifically, we specify a template which contains items such as the description of the scenario, the business goal purpose which is supported by an analyze purpose from the information need goal standpoint, the amount of involved quality views, an example of the evaluation scenario, the suitable strategy pattern to be instantiated regarding the purpose type and the amount of quality views.Besides, the template includes the generic process specification of the strategy pattern and the concrete evaluation strategy to be applied in order to achieve the operational business goal.
In the following subsections we document six evaluation scenarios.For all scenarios, the corresponding business goal purpose can be achieved by using the suitable strategy, which is driven by measurement, evaluation, analysis and, when necessary, change activities.So in subsection 5.1, we consider the scenario for the understanding purpose and one quality view; in subsection 5.2, we describe the scenario for the improving purpose and one quality view.Also, the situation for the same improving purpose but for two related quality views is documented in subsection 5.3.In subsection 5.4, we consider the scenario for the monitoring and controlling purpose for one quality view.Then, in subsection 5.5, we document the scenario for the selection of one alternative from a set of competitive entities.Finally, the scenario for comparing competitive entities and adopting the best capabilities or characteristics to a target entity is described in subsection 5.6.At the end of each subsection we summarize some specific issues for a better comprehension of the evaluation scenario.Additionally, in subsection 5.7, we highlight some general issues for all of them.
Ultimately, the illustration of these scenarios can help the reader to be aware what the S.4.1 ("Select a strategy pattern for each ME/MEC project") step involves considering business goal purposes.

Evaluation Scenario for the Understanding Purpose
Description: The business goal purpose at operational level is to understand the current state of an entity, in a given context, for a set of characteristics and attributes related to a quality focus, through the systematic use of an understanding strategy driven by measurement, evaluation and analysis activities.The measurement activity is performed by quantifying attributes by means of the selected metrics.The evaluation activity is performed by interpreting characteristics and attributes by means of indicators.The analysis is based on determining strengths and weaknesses of the evaluated entity in a given moment, which produces a conclusion and recommendation report.

Business Goal Purpose: Understand
Amount of Quality Views: One.Note: Table 4 shows examples of quality views

Example of Evaluation Scenario:
• Business Goal statement: Understand the Security level of the SIU Guaraní System School at UNLPam (Universidad Nacional de La Pampa) • Entity Category: System • Concrete Entity: SIU Guaraní, in the context of the • Quality Focus: External Quality • Characteristic: Security (Subcharacteristics: • Quality View: System Quality View Strategy Pattern to be instantiated: View) [38] Process Specification in GoME_1QV:

GoME_1QV pattern
Strategy to be applied: GOCAME (Goal-Oriented Context-Aware Measurement and Evaluation).This strategy was used in [28] for the exemplified Security evaluation scenario.
In Fig. 5 we have specified the behavioral and functional process perspectives for the GoME_1QV strategy pattern.The specification shows sequences, parallelisms in conjunction with inputs and outputs of the activities.In order not to clutter the process model with all this information, we use an abridged process specification (see Fig. 6) which considers just the behavioral process perspective for GoME_1QV.Also, it includes the business goal with its purpose and the information need goal -whose purpose is always analyze.Hereafter, in the following scenarios, we just consider the abridged process specification for each pattern.Besides, in order to reduce the complexity for the reader, "Design the Measurement" and "Design the Evaluation" activities from Fig. 5 were integrated into the A2 activity in Fig. 6 (i.e., the "Design Measurement and Evaluation").Likewise, "Implement the Measurement" and "Implement the Evaluation" activities from Fig. 5 were integrated into the A3 activity (i.e., the "Implement Measurement and Evaluation").
The A4 ("Analyze Results") activity permits to draw conclusions and recommendations about the strengths and weaknesses of the evaluated entity in a given moment, just like a snapshot analysis of the current state of the evaluated entity.Usually, in this scenario, the frequency of ME (A3 activity) is not an issue, as does it is in the monitoring and controlling scenario, which is illustrated in subsection 5.4.

Evaluation Scenario for the Improving Purpose
Description: The business goal purpose at operational level is to understand and improve the current state of an entity, in a given context, for a set of characteristics and attributes related to a quality focus, through the systematic use of an improvement strategy driven by measurement, evaluation, analysis and change activities.The measurement activity is performed by quantifying attributes by means of the selected metrics.The evaluation activity is performed by interpreting characteristics and attributes by means of indicators.The analysis is based on determining weaknesses or vulnerabilities of the evaluated entity in a given moment, which produces a conclusion and recommendation report about opportunities for improvement.The improvement can be achieved by means of changes in the entity and/or in its context.Once the changes were performed, the new entity version (and/or context) is re-evaluated for analyzing the actual impact and gain of the improvement.

Business Goal Purpose: Improve
Amount of Quality Views: One.Note: Table 4 shows examples of quality views

Example of Evaluation Scenario:
• Business Goal statement: Improve 10% the Facebook mobile application Usability in 6 months Notice that this pattern is fully documented in Appendix I.
Abridged Process Specification: Strategy to be applied: GOCAMEC ( strategy was used in [35,38] for the exemplified was also used as proof of concept to illustrate Unlike the GoME_1QV strategy pattern entity version (see A5 and A6 activities in Fig. 7).Also from A6 to A3 activities.Changes are designed and implemented according given in the A4 activity.Aimed at knowing the should be performed.If the planned improvement gain on quality after i.e., the operational business goal was evaluation cycles should be performed.
As commented in Section 4, the process pattern specification allows to indentify concrete subgoals useful for realizing the S.4.2 ("Per each selected strategy pattern, identify the concrete ME information need/business goals from the pattern process specification three subgoals viz.(i) understand the current quality state of the entity make changes on it -by performing the A5 and A6 activities improvement) after changes -by performing again the A3 a Fig. 3 for the Facebook case.

Another Evaluation Scenario for the
Description: Considering that a given quality view depends on 'independent view' roles specified in the state of a concrete entity belonging to a dependent quality view by applying evaluation entity belonging to a related independent quality view.Hence, t understand and ultimately improve the current attributes for a dependent view's quality focus by applying evaluation independent view's quality focus, through the systematic use of an improvement strategy driven by measurement, evaluation, analysis and change activities.The measurement activity is the quantification of attributes of the entity belonging to th of attributes of the entity belonging to the independent view.(Goal-Oriented Context-Aware Measurement, Evaluation and Change ] for the exemplified Usability evaluation scenario.Note that this evaluation scenario was also used as proof of concept to illustrate some steps for the Quality Evaluation Approach in pattern, the GoMEC_1QV pattern includes activities devoted to develop a new entity version (see A5 and A6 activities in Fig. 7).Also, it includes re-evaluation cycles as depicted by .Changes are designed and implemented according to the improvement recommendations given in the A4 activity.Aimed at knowing the quality impact produced after changes on the entity the planned improvement gain on quality after the implemented changes was not achieved considering the information need, then new change and re As commented in Section 4, the process pattern specification allows to indentify concrete subgoals Per each selected strategy pattern, identify the concrete ME information need/business goals from the pattern process specification") step.Particularly, the GoMEC_1QV pattern establishes d the current quality state of the entity -by performing the A1 to A4 activities; by performing the A5 and A6 activities; and (iii) understand the ulterior quality state (the by performing again the A3 and A4 activities.These subgoals were also illustrated in

Evaluation Scenario for the Improving Purpose
Considering that a given quality view depends on another quality view (see the 'dependent view' and 'independent view' roles specified in the quality view component of Fig. 1), the purpose is to improve the current of a concrete entity belonging to a dependent quality view by applying evaluation-driven changes to other entity belonging to a related independent quality view.Hence, the business goal purpose at operational level the current state of an entity, in a given context, for a set of characteristics and quality focus by applying evaluation-driven changes to other entity through the systematic use of an improvement strategy driven by measurement, evaluation, analysis and change activities.The measurement activity is performed at least twice the quantification of attributes of the entity belonging to the dependent view; Second, by means of the quantification of attributes of the entity belonging to the independent view.The evaluation activity is performed characteristics and attributes by means of indicators.The analysis is based on of characteristics and attributes related to the dependent view.Those benchmarked indicators of the dependent view permit to identify problems and derive related attributes As commented in Section 4, the process pattern specification allows to indentify concrete subgoals which are Per each selected strategy pattern, identify the concrete ME information the GoMEC_1QV pattern establishes forming the A1 to A4 activities; (ii) and (iii) understand the ulterior quality state (the These subgoals were also illustrated in other quality view (see the 'dependent view' and component of Fig. 1), the purpose is to improve the current driven changes to other at operational level is to in a given context, for a set of characteristics and driven changes to other entity for an through the systematic use of an improvement strategy driven by measurement, twice: First, by means of econd, by means of the quantification rmed at least twice as .The analysis is based on of characteristics and attributes related to the dependent view.Those weakly attributes for the entity of the independent view.For the independent view, the analysis proposes recommendations for change.When the changes were performed in the entity of the independent view, the dependent view's entity is therefore re-evaluated for analyzing the improvement gain.

Business Goal Purpose: Improve
Amount of Quality Views: Two.Note: Table 4 shows examples of quality views.Also, Fig. 8 shows 'influences' and 'depends on' relationships between typical quality views in software development and evaluation projects.The direction of the 'influences' relationship is from the independent quality view to the dependent quality view.
Conversely, the direction of the 'depends on' relationship is from the dependent quality view to the independent quality view.and Learnability in use).Note: The characteristics related to the independent quality view are derived from problems detected in the JIRA system-in-use evaluation.In the case study performed in [22], the derived characteristics were Usability and Information Quality for the External Quality focus.
• Quality Views: System-in-Use Quality View and System Quality View Strategy Pattern to be instantiated: GoMEC_2QV (Goal-oriented Measurement, Evaluation and Change for Two Quality Views) [38] Abridged Specification for the Generic Process: At first, the SIQinU strategy was conceived for t However, the GoMEC_2QV strategy pattern can be applied for any couple of related for a better comprehension of Fig. 9, we comment the JIRA evaluation scenario of a real company [22].In this case, two quality views the System Quality View.Starting with "entering a new defect" task and context of use were specified characteristic, and attributes combined to Effe established as nonfunctional requirements.
In the A2_DQV activity, metrics and indicators were designed yielding measure and indicator values for Quality in Use acceptability levels achieved Results for the DQV") activity we found that some Actual Usability attributes were not satisfied.derive External Quality attributes -the independent quality view System-in-Use Quality View 'depends on' as shown in Fig. 8).Specifically, in A1_IQV we specified subcharacteristics and attributes related to Usability and Information Quality for the External Quality focus.Then In A4_IQV, we analyzed and proposed recommendations for benchmarked External Quality indicators.and performed (A5_IQV and A6_IQV point of view.Finally, we re-evaluated A3_DQV activity again.So the Quality in Use In summary, we were able to gauge how the improvements made in the JIRA web application (the GoMEC_2QV process specification re- Strategy for understanding and Improving Quality in Use).This strategy evaluation scenario.

Evaluation Scenario for the
he SIQinU strategy was conceived for those yellow-colored quality views highlighted i strategy pattern can be applied for any couple of related quality view for a better comprehension of Fig. 9, we comment the JIRA evaluation scenario which was performed in two quality views were involved namely the System-in-Use tarting with the Quality in Use focus -the dependent quality view task and context of use were specified in the A1_DQV activity.Also, the Actual Usability characteristic, and attributes combined to Effectiveness, Efficiency, and Learnability-in-use subcharacteristics were established as nonfunctional requirements.and indicators were designed.Then, in A3_DQV the ME values for characteristics and attributes.The results allowed us to understand the achieved and to perform a preliminary analysis.During the A4 we found that some Actual Usability attributes were not satisfied.
independent quality view-that can influence on Quality in Use 'depends on' the System Quality View, and in turn the latter 'influences' the former, Specifically, in A1_IQV we specified subcharacteristics and attributes related to Usability and Information Quality for the External Quality focus.Then, the A2_IQV and A3_IQV activities were carried out.
we analyzed and proposed recommendations for changes in the JIRA system, benchmarked External Quality indicators.Once changes, using re-parameterization as change method activities), the new JIRA version was re-evaluated from the External Quality evaluated the Quality in Use of the new JIRA in the same context Quality in Use improvement gain was determined in the A4_DQV activity how the quality of JIRA in use (the dependent quality view JIRA web application (the independent quality view).It is important to remark that in -evaluation cycles are allowed for the two involved quality views.

Evaluation Scenario for the Monitoring and Controlling Purpose
business goal purpose at operational level is to monitor and control the state This strategy was used colored quality views highlighted in Fig. 8. quality views.In the sequel, which was performed in the context Use Quality View and the dependent quality view-, the evaluated Also, the Actual Usability use subcharacteristics were ME were performed allowed us to understand the A4_DQV ("Analyze we found that some Actual Usability attributes were not satisfied.This enabled us to Quality in Use.(Note that the 'influences' the former, Specifically, in A1_IQV we specified subcharacteristics and attributes related to Usability and IQV activities were carried out. in the JIRA system, regarding weakly parameterization as change method were designed evaluated from the External Quality in the same context performing the in the A4_DQV activity.dependent quality view) was enhanced by It is important to remark that in the quality views.
state of an entity, in a given context, for a set of characteristics and attributes related to a quality focus, through the systematic use of a monitor and control strategy driven by measurement, evaluation, analysis and, if necessary, change activities.measurement activity is performed by quantifying attributes by means of the selected metrics.activity is performed by interpreting characteristics and attributes by means of indicators and control variables).The analysis activity is base supervise (control) deviations with respect to the established identify preventive and corrective actions and/or to determine actions imply changes in the entity and/or its context.The monitor and control of detected performance problems were resolved o

Business Goal Purpose: Monitor and Control
Amount of Quality Views: One.Note: Table 4 shows examples of quality views

Example of Evaluation Scenario:
• Business Goal statement: Monitor and control that the Guaraní system is assured over time In this evaluation scenario, the A4 ( described in Fig. 6 and Fig. 7 for Particularly, in Fig. 10, the A4 activity is based on continuo monitoring and control tasks with somewhat goal.Monitoring and control allow to supervise CLEI ELECTRONIC JOURNAL, VOLUME 19, NUMBER 3, PAPER 3, DECEMBER 2016 19 given context, for a set of characteristics and attributes related to a quality focus, through the systematic use of a monitor and control strategy driven by measurement, evaluation, analysis and, if necessary, change activities. s performed by quantifying attributes by means of the selected metrics.activity is performed by interpreting characteristics and attributes by means of indicators (also named and control variables).The analysis activity is based on a continuous ME and critical observation (monitor) to respect to the established acceptability levels of indicators actions and/or to determine predictions and trends.The preventive and corrective actions imply changes in the entity and/or its context.The monitor and control of an entity permit detected performance problems were resolved or if new change actions are necessary.

Monitor and Control
: Table 4 shows examples of quality views to say, a realization of business/information need (performance variables) are under control regarding the expected acceptability levels.The monitoring and control loop includes the A3 and A4 is, there is a continuous cycle of implementing the measurement and evaluation again and again.Furthermore, preventive or corrective are detected in the A4 activity.If this is the case, changes should be performed in the A5 and A6 activities.Therefore, the follow up cycles of monitoring and control In this scenario, different kind of analysis can be performed such as performance, critically observing or determining the status, determine and assess trends, among others.

Evaluation Scenario for the
Description: The business goal purpose competitive entities, in a given context, for a set of characteristics and attributes related to a quality focus, through the systematic use of a selection strategy driven by measurement, evaluation and anal is performed by quantifying attributes by means of the selected metrics.The evaluation activity is performed by interpreting characteristics and attributes by means of indicators.The analysis is based on determining strengt weaknesses of the evaluated entities in a given moment, which allows selecting and adopting the best alternative for the established quality focus.

Business Goal Purpose: Select an alternative
Amount of Quality Views: One.Note: Table 4 shows examples of quality views

Example of Evaluation Scenario:
• Business Goal statement: Select the best academic management system for the UNLPam, which ensures quality criteria such as Functional Suitability, Efficiency, Usability, • Entity Category: System • Concrete Entities: SIU Guaraní, webSIA cycle of implementing the measurement and evaluation, and then analyzing and controlling rmore, preventive or corrective actions could be performed concurrently, whether If this is the case, changes should be performed in the A5 and A6 activities.monitoring and control take into account the new entity.In this scenario, different kind of analysis can be performed such as assessing actual performance with planned determining the status, predictive and corrective analysis ng others.

Evaluation Scenario for the Selection Purpose
business goal purpose at operational level is to select the best alternative among a set of competitive entities, in a given context, for a set of characteristics and attributes related to a quality focus, through the systematic use of a selection strategy driven by measurement, evaluation and analysis.The measurement activity is performed by quantifying attributes by means of the selected metrics.The evaluation activity is performed by interpreting characteristics and attributes by means of indicators.The analysis is based on determining strengt in a given moment, which allows to analyze and compare them with the aim of the best alternative for the established quality focus.

Select an alternative
: Table 4 shows examples of quality views Select the best academic management system for the Engineering School at quality criteria such as Functional Suitability, Efficiency, Usability, SIU Guaraní, webSIA 3  Unlike other strategy patterns' process specifications described before, the first activity in GoMES_1QV is the A0 activity.This activity is named "Preselect Competitive Entities" in Fig. 11.It consists in filtering and determining the set of competitive entities to be evaluated.Preselection can be based for instance on expert judgment, or on more objective protocols and selection criteria.The A0 primary aim is to reduce the sample to those relevant and suitable competitive entities.
After the preselection is performed, the "Define non Functional Requirements" and "Design Measurement and Evaluation" activities follow up in the workflow.Note that A1 and A2 activities are the same as the ones previously commented for the other strategy patterns.Likewise, the A3 ("Implement Measurement and Evaluation") activity is the same as before but performed in an iterative way.Hence A3 must be carried out per each competitive entity that have been preselected, producing as outcome measure and indicator values per each entity regarding the same evaluated nonfunctional requirements.
Consequently, the A4 ("Analyze Results and Select Alternative") activity permits to compare the evaluation results for the competitive entities with the aim of selecting the best alternative.In this sense, the analysis is based on determining strengths and weaknesses of the evaluated entities, which allows to analyze and compare them with the aim of selecting and adopting the highest scored alternative for the established quality focus and requirements.
In a nutshell, the GOCAMES strategy fosters a systematic way to get data and information of competitive entities, which ultimately are the inputs for sound analysis.By means of this analysis, the information need gives support to the operational business goal whose purpose is "select alternative".
As a final remark, another evaluation scenario for the selection purpose which considers both a quality view and a cost view can be specified.In this new scenario, the requirements should be established regarding quality characteristics and attributes in addition to cost factors.Moreover, the analysis and selection model should consider specific quality-cost indicator relations.

Evaluation Scenario for the Comparing Purpose
Description: The business goal purpose at operational level is to compare characteristics and attributes of a set of representative entities, in a given context, aimed at incorporating (adopting) recommended strengths to a new entity or to an existing one, through the systematic use of a comparison strategy driven by measurement, evaluation, analysis and, when necessary, change.The measurement activity is performed by quantifying attributes by means of the selected metrics.The evaluation activity is performed by interpreting characteristics and attributes by means of indicators.The comparative analysis is based on determining strengths and weaknesses of the evaluated entities in a given moment, which allows recommending the detected strengths and adopting them in a new entity or in one that already exists.

Business Goal Purpose: Compare and Adopt
Amount of Quality Views: One.Note: Table 4   Similarly to the GoMES_1QV process specification (described in the previous subsection) process starts with the preselection (A0) the preselection of representative entities can be based on expert judgment, on existing benchmarks or case studies, or on other agreed preselection criteria.that we are interested in improving it.
As it occurs in GoMES_1QV, the A3 per each preselected representative entity, producing as outcome measure and indicator evaluated nonfunctional requirements.
The A4 ("Analyze Results and Recommend Strengths to be Adopted" one described for the purpose of selecti best ranked characteristics and attributes entity already exists -as it was the situation in the strengths should therefore be designed Conversely, if at that moment the entity does not exist, then A5 and A6 activities are not development strategy should be chosen capabilities or characteristics that have been detected.

Final Remarks about Evaluation Scenarios
As additional remarks about the above described scenarios, we would like to point out two aspects.Firstly, there are evaluation purposes which are aimed at better characterizing and understanding the current sta an entity, without influencing it (i.e., without introducing changes in the entity) purposes which are aimed at influencing it the understanding purpose (subsection 5.1) control and change activities are not necessary due to the scope of the for selecting one alternative from a set of competitive entities (subsection 5.5) does not imply a change entity, but rather its adoption and installation for the improving purpose (subsections 5.2 and 5.3) which always implies changes purpose (subsection 5.4), in which monitor, comparing competitive entities and adopting the best capabilities or characteristics to a target entity may imply changes as described in subsection 5.6.
Secondly, we can observe a high level of activity reuse looking at the abridged process specifications of strategy patterns for the illustrated evaluation scenarios -even for the evaluation scenario that : Behavioral process perspective for the GoMECom_1QV pattern GOCAMECom (Goal-Oriented Context-Aware Measurement, in [31] for the exemplified evaluation scenario. process specification (described in the previous subsection), th (A0) activity of representative entities to be compared, as seen the preselection of representative entities can be based on expert judgment, on existing benchmarks or case studies, preselection criteria.It is important to note that one of the preselected entities could be the A3 ("Implement Measurement and Evaluation") activity must be carried out ve entity, producing as outcome measure and indicator values regarding the same "Analyze Results and Recommend Strengths to be Adopted") activity is to some extent selecting an alternative.But in this case, after comparing the evaluation results, the and attributes could be taken into account to be adopted in a new or existing entity.If the situation in the [31] case study-, the necessary changes for in ed and implemented.These activities are coded A5 and the entity does not exist, then A5 and A6 activities are not performed chosen for designing and implementing a new entity taking into account the best characteristics that have been detected.

Remarks about Evaluation Scenarios
additional remarks about the above described scenarios, we would like to point out two aspects.Firstly, there are evaluation purposes which are aimed at better characterizing and understanding the current state (i.e., without introducing changes in the entity), whereas there are other evaluation purposes which are aimed at influencing it to some extent.For the former category, we can mention purpose (subsection 5.1), and the monitoring purpose (not illustrated in this paper), in which control and change activities are not necessary due to the scope of the evaluated situation.Additionally, one alternative from a set of competitive entities (subsection 5.5) does not imply a change entity, but rather its adoption and installation in an organization.For the latter category, we can mention subsections 5.2 and 5.3) which always implies changes, and the monitoring monitor, control and change activities are intertwined.Also and adopting the best capabilities or characteristics to a target entity may imply we can observe a high level of activity reuse looking at the abridged process specifications of strategy illustrated evaluation scenarios.For example, all process specifications share the A1 relates two quality views, documented in subsection 5.3 VOLUME 19, NUMBER 3, PAPER 3, DECEMBER 2016 Aware Measurement, Evaluation and , the GoMECom_1QV seen in Fig. 12. Again, the preselection of representative entities can be based on expert judgment, on existing benchmarks or case studies, selected entities could be the entity ) activity must be carried out es regarding the same to some extent similar to the alternative.But in this case, after comparing the evaluation results, the in a new or existing entity.If the for incorporating the and A6 in Fig. 12. performed.In this case, a a new entity taking into account the best additional remarks about the above described scenarios, we would like to point out two aspects.Firstly, there are te or performance of there are other evaluation we can mention the scenario for purpose (not illustrated in this paper), in which Additionally, the scenario one alternative from a set of competitive entities (subsection 5.5) does not imply a change in the target For the latter category, we can mention the scenario monitoring and control Also, the scenario for and adopting the best capabilities or characteristics to a target entity may imply we can observe a high level of activity reuse looking at the abridged process specifications of strategy ons share the A1-A4 activities documented in subsection 5.3.While the A1-A3 activities are the same at task level for all scenarios, the A4 activity can vary slightly at task level depending on the specific evaluation purpose, as previously commented at the end of the subsections.Furthermore, for those evaluation purposes which embrace changes, the A5-A6 activities can be reused totally.Note that for example while activity specifications can be the same for designing and implementing changes, different methods can be applied such as programming, refactoring, re-structuring and re-parameterization, among others.On the other hand, the A0 activity can be reused totally in those evaluation scenarios with preselection endeavors.
It is important to remark that even though many activities are reused in the process specifications of all evaluation scenarios, the dynamic of activities slightly differs each other, as shown in the behavioral process perspective respectively.Ultimately, the sound specification of evaluation scenarios for different evaluation purposes helps to know what to do and how to perform activities and methods in a systematic and disciplined way for achieving business goals at operational level by using strategies accordingly.

Conclusion and Future Work
In this scientific article, we have presented a holistic quality multilevel and multipurpose evaluation approach which considers the linkage of information need goals with business goals at different organizational levels such as operational, tactical and strategic in addition to different evaluation goal purposes.
Summarizing the first contribution listed in the Introduction Section, we have enlarged the conceptual base of the Quality Evaluation Approach by adding concepts such as strategy pattern and organizational level types in the business goal and project components.We have discussed the enhanced conceptual base in Section 3, which is able to relate business and information need goals with ME information needs, entity categories, quality focuses, quality views as well as with projects, strategies and strategy patterns.It is worthwhile remarking that this conceptual base is structured in subontologies.Considering the ontology scope, the terms included in the goal and project components are the minimum and necessary ones for describing goals, projects, strategies and strategy patterns.Also, they are aimed at adding conceptual robustness to our approach in addition to the ability to support semantic processability, among other benefits.
Regarding the second contribution, we have also defined in Section 4 the step by step of the Quality Evaluation Approach which lists the necessary activities to establish goals and projects at different organizational levels.In addition, we have illustrated the stepwise applicability using an evaluation scenario for improving the Facebook mobile app.Although there are some relevant references for the business goal alignment, as those analyzed in the Related Work Section, the approach we have proposed formally establishes the use of integrated ME/MEC strategies for helping to achieve both business and information need goals, mainly at operational level.Besides, the Quality Evaluation Approach specifies ME information need goals as well, which are linked to information need and business goals.Moreover, this approach fosters the use of strategy patterns as a reusable solution for instantiating the suitable evaluation strategy considering the project's goal statement and the strategy pattern intent.
Lastly, taking into account the third contribution, we have documented in Section 5 a set of six different evaluation scenarios for business/information need goals at operational level supported by strategy patterns and strategies.Specifically, we have identified different ME/MEC strategy patterns and strategies that realize purposes aimed at understanding, improving, monitoring and controlling, comparing and selecting entities regarding also the amount of quality views.Each evaluation scenario was illustrated with a concrete example.But the reader can envision that many examples can be applied for the same purpose considering the same or different quality views.Ultimately, the evaluation scenarios documented for business/information need goals at operational level, can be reused not only for the Software Engineering discipline but also to other disciplines such as Health Informatics, Biotechnology, Ecology, amongst many others.Finally, the established business goal purposes at operational level can be linked and provide feedback to business and information need goals at higher organizational levels.
Considering the semantic processability, an ongoing work is the development of a strategy pattern recommender system as a practical use of subontologies.This recommender system can be useful when an organization establishes ME/MEC projects.Hence, considering the project's goal statement and the strategy pattern intent, the recommender system will suggest the suitable strategy pattern that fits better to a given project.Consequently, the particular strategy will be easier to be customized.The process specification is made up from different perspectives, i.e., functional which includes activities, , which includes parallelisms, iterations, etc.; organizational agents, roles and responsibilities; and informational, which includes the structure and interrelations artifacts produced or consumed by activities.Considering the functional and behavioral perspective, Fig depicts the generic process for this pattern.The names of the eight MEC activities must be customized t the concrete quality view to be evaluated.
: Generic process from the functional and behavioral perspectives for the GoMEC_1QV pattern The method specification indicates how the descriptions of MEC activities must be performed.Tables 2 exemplify three method specification templates: one for a direct metric used as method specification for direct measurement tasks; one for an indirect metric, used in indirect measurements; and other for an elementary indicator, used in elementary evaluations.Note that terms in method specification templates come from the ME conceptual base.Many other method specifications can be envisioned such as task usage log files, questionnaires, aggregation methods for derived evaluation, amongst others.For change activities traditional structuring, re-parameterization, among others can be specified as well.
: Method specification templates for the Measurement task: a) for a Direct Metric; b) for an Indirect Metric

Direct Metric Template
Author: Version:

Figure 1 :
Figure 1: Key concepts from the business goal domain (business goal package) and relationships with some concepts for the project and nonfunctional requirements packages.Note: PO means Process Ontology; ME, Measurement and Evaluation; and MEC, Measurement, Evaluation and Change

Figure 2 :
Figure 2: Main instantiated activities of the Quality Evaluation Approach for the Facebook mobileapp scenario, using BPMN 4.3 step, we use the GOCAMEC ( strategy that instantiates the GoMEC_1QV strategy pattern.GOCAMEC specifications defined in this pattern (see Fig. I.2, and tables I.1 and I.2

Figure 3 :
Figure 3: Scenario instantiation where a Business Goal from the strategic level is decomposed in subgoals which are supported by Information Need Goals and ME Information Need Goals.Note: FmApp means Facebook mobile Application and ME, Measurement and Evaluation 4.3 step, we use the GOCAMEC (Goal-Oriented Context-Aware Measurement, Evaluation and Change strategy that instantiates the GoMEC_1QV strategy pattern.GOCAMEC customizes the process an cations defined in this pattern (see Fig. I.2, and tables I.1 and I.2, in Appendix I).Scenario instantiation where a Business Goal from the strategic level is decomposed in subgoals which are Goals and ME Information Need Goals.Note: FmApp means Facebook mobile Application and ME, Measurement and Evaluation VOLUME 19, NUMBER 3, PAPER 3, DECEMBER 2016

Figure 4 :
Figure 4: Instantiation of terms for the quality_view and non Information Need Goal.Note: FmApp means Facebook mobile Application and ME, Measurement and Evaluation

VOLUME 19 ,
NUMBER 3, PAPER 3, DECEMBER 2016 functional requirements components for the given ME Note: FmApp means Facebook mobile Application and ME, Measurement and Evaluation

Figure 5 :Figure 6 :
Figure 5: Functional and behavioral process perspectives for the Abridged Process Specification:

Figure 7 :
Figure 7: Behavioral process perspective for the GoMEC perspectives for this pattern is shown in Fig. I.2 T well.Evaluation interprets characteristics and attributes by means of indicators determining strengths and weaknesses of characteristics and attributes related to the dependent view.Those benchmarked indicators of the dependent view process perspective for the GoMEC_1QV pattern.The functional and behavioral process perspectives for this pattern is shown in Fig. I.2

VOLUME 19 ,
NUMBER 3, PAPER 3, DECEMBER 2016 functional and behavioral process Aware Measurement, Evaluation and Change).This evaluation scenario.Note that this evaluation scenario pproach in Section 4. includes activities devoted to develop a new as depicted by the flow provement recommendations on the entity, a re-evaluation changes was not achieved, new change and re-

Figure 8 :
Figure 8: An instantiation of typical quality views in software development and evaluation projects Example of Evaluation Scenario: • Business Goal statement: Improve the Quality in Use of the JIRA 2 considering changes in the JIRA system in 4 months time frame • Entity Category for the Dependent View: System in Use • Entity Category for the Independent View: System • Concrete Entities: JIRA in use, in the context of the ABC company, and JIRA web application • Quality Focuses: Quality in Use and External Quality • Characteristics of the Quality-in-use focus: Actual Usability (Subcharacteristics: Effectiveness, Efficiency,and Learnability in use).Note: The characteristics related to the independent quality view are derived from problems detected in the JIRA system-in-use evaluation.In the case study performed in[22], the derived characteristics were Usability and Information Quality for the External Quality focus.

Figure 9 :
Figure 9: Behavioral process perspective for the View, and IQV means Independent Quality View Strategy to be applied: SIQinU (Strategy for in the exemplified JIRA evaluation scenario.

Description:
The business goal purpose CLEI ELECTRONIC JOURNAL, VOLUME 19, NUMBER 3, PAPER 3, DECEMBER 2016 18 : Behavioral process perspective for the GoMEC_2QV pattern.Note that DQV means Dependent Quality View, and IQV means Independent Quality View

Figure 10 :
Figure 10: Behavioral process perspective for the Strategy to be applied: GOCAMEM Control).The use of this strategy was not published yet evolution of the Facebook mobileapp's user interface Usability onitor and control that the required acceptability level of Security for the ured over time, in the context of the Engineering School at UNLPam SIU Guaraní, in the context of the Engineering School at UNLPam External Quality (Subcharacteristics: Confidentiality, Integrity, Authenticity, and View Strategy Pattern to be instantiated: GoMEM_1QV (Goal-oriented Measurement, Evaluation Behavioral process perspective for the GoMEM_1QV pattern MC (Goal-Oriented Context-Aware Measurement, Evaluatio .The use of this strategy was not published yet, but we are currently monitoring thru the last three years the evolution of the Facebook mobileapp's user interface Usability.A4 ("Analyze and Control Results") activity differs to a some extent for the GoME_1QV and GoMEC_1QV process specifications activity is based on continuous observations, that is to say monitoring and control tasks with somewhat high frequency, depending on the specific business/ to supervise if the results of indicators (performance variables) are under control VOLUME 19, NUMBER 3, PAPER 3, DECEMBER 2016 given context, for a set of characteristics and attributes related to a quality focus, through the systematic use of a monitor and control strategy driven by measurement, evaluation, analysis and, if necessary, change activities.The s performed by quantifying attributes by means of the selected metrics.The evaluation also named performance observation (monitor) to indicators which allow to .The preventive and corrective permit analyzing if the required acceptability level of Security for the SIU Engineering School at UNLPam Availability) oriented Measurement, Evaluation, Monitor and Aware Measurement, Evaluation, Monitor and monitoring thru the last three years the to a some extent from those process specifications respectively.

Figure 11 :
Figure 11: Behavioral process perspective for the and UCASAL-SAG 4 External Quality Functional Suitability, Efficiency, Usability and Security System Quality View GoMES_1QV (Goal-oriented Measurement, Evaluation and Selection for : Behavioral process perspective for the GoMES_1QV pattern http://sistemas.ucasal.edu.ar:7779/ords/f?p=102:1:7937987068216VOLUME 19, NUMBER 3, PAPER 3, DECEMBER 2016 acceptability levels.The monitoring and control loop includes the A3 and A4 activities.That analyzing and controlling , whether deviations If this is the case, changes should be performed in the A5 and A6 activities.actual performance with planned analysis, in addition to is to select the best alternative among a set of competitive entities, in a given context, for a set of characteristics and attributes related to a quality focus, through ysis.The measurement activity is performed by quantifying attributes by means of the selected metrics.The evaluation activity is performed by interpreting characteristics and attributes by means of indicators.The analysis is based on determining strengths and them with the aim of Engineering School at quality criteria such as Functional Suitability, Efficiency, Usability, and Security oriented Measurement, Evaluation and Selection for Strategy to be applied: GOCAMES (Goal-Oriented Context-Aware Measurement, Evaluation and Selection).The use of this strategy was not published yet.

Figure 12 :
Figure 12: Behavioral process perspective for the Strategy to be applied: GOCAMECom Comparison).This strategy was used in [ II.The process specification is made up from different perspectives, i.e., inputs, outputs, etc.; behavioral, which includes parallelisms, iterations, etc.; agents, roles and responsibilities; and artifacts produced or consumed by activities.Considering the functional and behavioral perspective, Fig depicts the generic process for this pattern.The names of the eight MEC activities must be customized strategy taking into account the concrete quality view to be evaluated.

Figure I. 2 :Table I. 1 :
Figure I.2: Generic process from the functional and behavioral perspectives for the GoMEC_1QV pattern III.The method specification indicates how the descriptions of MEC activities must be performed.Tables I.2 exemplify three method specification templates: one for a direct metric used as method specification for direct measurement tasks; one for an indirect metric, used in indirect measurements; and other for an elementary indicator, used in elementary eva from the ME conceptual base.Many other method specifications can be envisioned such as task usage log files, questionnaires, aggregation methods for derived evaluation, amongst others.For c methods such as refactoring, re-structuring, re Table I.1: Method specification templates for the Measurement task: a) for a Direct Metric; b) for an Indirect

[
about the used tool if any) b) Indirect Metric Template Quantified Attribute name: Metric name: Objective:

Table 2 :
[4],21]te The rationale for achieving a specified Goal.Note: Examples of evaluation purposes are: understand, improve, monitor and control, compare and select, predict.timeframeAsetperiod of time in which the Goal is pursued or is expected to be achieved.Note:A timeframe of a Goal can range from short-, intermediate-term to long-term period of time.Organization mission It states the organization's core purpose and focus.Note: A mission statement normally remains unchanged over time.visionItdescribeswhatanorganization wants to aspire to, and what specific motivation binds together the organization's stakeholders.Note: A vision reflects the realization of the organization's values.Organizational Level levelIt represents a specific level in which an Organization's Business Goals are formulated and Information Need Goals are taken into account.Note 1: Commonly, three levels are identified in the literature such as STRATEGIC, TACTICAL and OPERATIONAL levels[17,21].Regarding[4]different goals exist at different levels of an organization such as the management level, the department level, the project level.These levels match to a great extent with the above mentioned levels.
definitions for the business goal and project components Term Attribute Definition Goal statement An explicit declaration of the aim to be achieved.Note: A statement is usually a written assertion in a high-level or natural language.purpose Note 2: Usually, a long-term Business Goal is formulated at a Strategic Level, while a short-or intermediate-term Business Goal is formulated at an Operational or Tactical Level.Project name Label or name of a Project to be identified.startDate Date or instant of time when a Project is started.endDate Date or instant of time when a Project is finished.Strategy name Label or name of the Strategy to be identified.Project Life Cycle phases A phase of a Project Life Cycle is a group of strongly-related processes and activities defined in a given order.Note 1: A phase ends with a milestone, such as the completion of a deliverable like a project plan.Note 2: A Project Life Cycle has many phases such as Planning, Scheduling, Monitoring, among others.Strategy Pattern name A descriptive and unique name.Note: The label or name is usually expressed in English.alias Acronym or other names for the strategy pattern.intent Main objective for the strategy pattern.motivation Project problem/goal solved by the strategy pattern.

Table 3 :
Relationship definitions for the business goal and project componentsRelationship Definition adoptsA Project Management process adopts a Project Life Cycle.arrangesWorkBy An Organization organizes its work or effort by means of Projects for the achievement of its established Goals.
establishes An Organization establishes and pursues Goals as part of its Mission, and in alignment with its Vision.helpsToAchieve A Strategy gives support for achieving one or more organizational Goals.instantiates A Strategy can instantiate none or one Strategy Pattern.involves The realization of a Project Life Cycle involves Resources and Work Definitions, among other aspects.isFormulatedAt A Business Goal is formulated at one Organizational Level.isManagedAt An Organization is managed at different Organizational Levels, such as strategic, tactical and operational levels.isManagedBy A Project is managed by means of a Project Management process.isSupportedBy A Business Goal is supported by none or several Information Need Goals.operationalizes A Project operationalizes Goals.produces A Project Management process produces a Project Management Plan as artifact.requires An Information Need Goal requires none or several ME Information Needs.subGoal A Business Goal can be divided into subgoals, which are in turn Business Goals at lower granularity levels.uses A Project Life Cycle uses one or more Strategies.
1. Establish a business goal at any organizational level..4.1.Select a strategy pattern for each ME/MEC project.For this selection, look at the amount of quality views and purpose involved in the project's goal statement.S.4.2.Per each selected strategy pattern, identify the concrete ME information need/business goals from the pattern process specification.S.4.3.Instantiate a strategy appropriately from each strategy pattern.Also, schedule this resource into the project life cycle accordingly.
S.2.Refine the business goal, if necessary, into tactical/operational business goals.S.3.Establish information need goals for each business goal at the corresponding organizational level.S.4.Formulate ME/MEC projects for those goals that require ME activities.SS.5.Perform the ME/MEC projects.S.6.Check the achievement of business goals by analyzing information need goals.

Table 4 :
Quality Views examples in Software Engineering's production lines

shows examples of quality views Example of Evaluation Scenario:
Strategy Pattern to be instantiated: GoMECom_1QV (Goal-oriented Measurement, Evaluation and Comparison for One Quality View)Abridged Process Specification: