Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment Ecological Indicators

Sustainability assessments require the management of a wide variety of information types, parameters and uncertainties. Multi criteria decision analysis (MCDA) has been regarded as a suitable set of methods to perform sustainability evaluations as a result of its ﬂ exibility and the possibility of facilitating the dialogue between stakeholders, analysts and scientists. However, it has been reported that researchers do not usually properly de ﬁ ne the reasons for choosing a certain MCDA method instead of another. Familiarity and af ﬁ nity with a certain approach seem to be the drivers for the choice of a certain procedure. This review paper presents the performance of ﬁ ve MCDA methods (i.e. MAUT, AHP, PROMETHEE, ELECTRE and DRSA) in respect to ten crucial criteria that sustainability assessments tools should satisfy, among which are a life cycle perspective, thresholds and uncertainty management, software support and ease of use. The review shows that MAUT and AHP are fairly simple to understand and have good software support, but they are cognitively demanding for the decision makers, and can only embrace a weak sustainability perspective as trade-offs are the norm. Mixed information and uncertainty can be managed by all the methods, while robust results can only be obtained with MAUT. ELECTRE, PROMETHEE and DRSA are non-compensatory approaches which consent to use a strong sustainability concept, accept a variety of thresholds, but suffer from rank reversal. DRSA is less demanding in terms of preference elicitation, is very easy to understand and provides a straightforward set of decision rules expressed in the form of elementary “ if . . . then . . . ” conditions. Dedicated software is available for all the approaches with a medium to wide range of results capability representation. DRSA emerges as the easiest method, followed by AHP, PROMETHEE and MAUT, while ELECTRE is


Introduction
The literature about sustainability assessment is wide and steadily growing, with different interpretations and implementations of this concept available so far (Bond et al., 2012). On one side there are rather conceptual and holistic proposals based on sustainability principles (see for example WCED 1987; Gibson, 2006), which introduce frameworks to encompass and combine different values and perspectives, while one the other side there are more concrete and operational approaches that try to define and derive sustainability criteria/pillars to make the concept of sustainability operational (Omann 2004;Pope et al., 2004;Gibson, 2006;Cinelli et al., 2013a;Sala et al., 2013a). There are different attempts to perform this operationalization, ranging from two to seven pillars depending on the context of the analysis/evaluation to be performed (Gibson, 2006;Bond et al., 2012). One of the most common ones is the triple bottom line (TBL) approach, which is based on the environmental, economic and social pillars, having equal importance in the decision-making process (Pope et al., 2004;Gibson, 2006;Convertino et al., 2013;Subramanian et al., 2014;Tatham et al., 2014). This approach has been widely used as it fits properly with the professional figures and organizational bodies that are in charge of the assessment of each of the pillars (Gibson, 2006).
The objective of sustainability assessment (SA) can vary considerably, from a micro to a macro scale, meaning that the inclusion of various processes and mechanisms cannot always be taken into account with the same approaches (Cinelli et al., 2013b;Zamagni et al., 2009). This leads to the necessity to define clearly what the scope of the assessment is and what questions need to be answered, implying that different instruments should be used depending on each case (Sala et al., 2013b). Additionally, the spheres or pillars of sustainability considered can vary, which means that some studies can consider only environmental and economic aspects, others only the environmental ones and others environmental, economic and social together (Sala et al., 2013a).
SA has also the role of improving the decision aiding process, by Gasparatos et al., 2008): Integrating sustainability spheres and considering their interdependencies. Including intragenerational and intergenerational considerations. Supporting constructive interaction among stakeholders.
Accounting for uncertainties and adopting a precautionary approach. Contributing to monitoring and communication of results.
Over the past decades a plethora of methodologies and tools were developed to perform sustainability assessment studies, focusing on different scopes (i.e. different pillars) and scales/ objectives (i.e. micro, meso and macro), with some covering only a certain pillar and object of sustainability (e.g. life cycle assessment), and with others widening both (e.g. cost-benefit analysis, multi criteria decision analysis) (Singh et al., 2009;Zamagni et al., 2009). For example, life cycle assessment (LCA) is a productoriented tool for the assessment of environmental implications, while multi criteria decision analysis (MCDA) is a set of methods that can be used to compare alternatives from a product level to a policy one, by covering one or more sustainability pillars (Munda, 2005;Epa, 2006). Furthermore, Ness and coworkers (Ness et al., 2007) provide a categorization of sustainability assessment tools which includes (i) indicators which are non-integrated, (ii) product related assessments and (iii) integrated assessments. Non-integrated indicators support the decision-making processes by converting knowledge in manageable units of information (UN 2001). They can be defined as an "operational representation of an attribute of a system" (Gasparatos et al., 2008), which can be an environmental, economic or social state of the system under consideration. Some examples are the Environmental pressure indicators and the national indicators developed by United Nations Division for Sustainable Development (UN, 2001;Gasparatos et al., 2008). Product-related tools consider flows in relation to production and consumption of goods and services. An important distinction that can be introduced here and is applicable to all the tools for SA is the extend of the system tackled, in other words whether the method takes into account only direct impact of the target or is based on a life-cycle approach. Product energy analysis is an example of a tool covering only direct impacts, whereas LCA spans the whole life cycle stages of a product (De Ridder et al., 2007;Ness et al., 2007). Integrated assessment are all the approaches that try to handle the information from individual indicators in a comprehensive manner, by considering interrelations and interdependencies among them, accounting for the different importance that they might have, and adopting different degrees of aggregation. MCDA is one of this and it has been indicated as the appropriate set of tools to perform assessments of sustainability, by considering different sustainability spheres, perspectives, stakeholders, values, uncertainties and intra and inter-generational considerations (O'Neill et al., 1999;Munda 2005;Gasparatos et al., 2008;Bond et al., 2012;Gasparatos and Scolobig, 2012).
Due to the complexity of sustainability assessment and the need to provide a path towards the achievement of a sustainable future, decisions have to be taken and this must happen in a structured, transparent and reliable way and MCDA can contribute to such objective.
MCDA consists of a group of approaches which allow to account explicitly for multiple criteria, in order to support individuals or groups to rank, select and/or compare different alternatives (e.g. products, technologies, policies) (Belton and Stewart, 2002). A general grouping of MCDA approaches has been proposed by (Slowinski et al., 2002;Greco et al., 2004), who distinguish three underlying theories: (i) utility function, (ii) outranking relation and (iii) sets of decision rules. The utilitybased theory includes methods synthesizing the information in a unique parameter (also called performance aggregation based approaches) and it was introduced during the 1970s by (Keeney and Raiffa, 1976). The outranking relation theory involves methods based on comparisons between pairs of options to verify whether "alternative a is at least as good as alternative b" (also called preference aggregation based approaches) (Roy, 1991). The decision rule theory originates from the artificial intelligence domain and it allows deriving a preference model through the use of classification or comparison of decision examples (Greco et al., 2001b).
This review article builds upon previous analyses performed by (Cinelli et al., 2013a) and (Sadok et al., 2008) and investigates how MCDA can contribute to Sustainability Assessment by analyzing five MCDA methods on the basis of ten comparison criteria that they should satisfy to properly handle problems concerning sustainability. The methods from each of the three families mentioned above have been chosen to provide a wide coverage of MCDA: For the utility-based theory: multi attribute utility theory (MAUT) and analytical hierarchy process (AHP). For the outranking relation theory: elimination and choice expressing the reality (ELECTRE) and preference ranking organization method for enrichment of evaluations (PROME-THEE). For the sets of decision rules theory: dominance based rough set approach (DRSA).
The reasons for the selection on these sets of methods are that (i) utility-based (especially MAUT and AHP) and outranking approaches (specifically ELECTRE and PROMETHEE) have been the most widely MCDA tools in sustainability-related research as reported in various publications (Wang et al., 2009;Huang et al., 2011;Herva and Roca, 2013) and (ii) the decision rules technique has been proposed as a powerful instrument to conduct sustainability assessments (Sadok et al., 2008), suggesting the usefulness of a deeper analysis of its potentials in this area.
Sections 2 briefly reviews the selected MCDA methods and describes the research methodology that was used. Section 3 reports the evaluation of the methods on the basis of the ten comparison criteria, Section 4 discusses the results and the main conclusions can be found in Section 5.

MCDA methods
This sub-section provides a brief description of the main features of each MCDA method that has been analyzed, with a focus on the distinctive characteristics that are crucial for the analysis phase.

Multi attribute utility theory (MAUT)
Multi attribute utility theory is a performance aggregation based approach, which requires the identification of utility functions and weights for each attribute that can then be assembled in a unique synthesizing criterion (Keeney and Raiffa, 1993;Dyer 2005), with the additive and multiplicative aggregations being the most widely applied (De Montis et al., 2005).
The selection of the relevant aggregation procedure requires the verification of various assumptions, and a discussion about this important procedural step can be found in (Keeney 1974;Keeney and Raiffa, 1993;De Montis et al., 2005).

Analytical hierarchy process (AHP)
The analytic hierarchy process is another approach of the first performance aggregation based approaches and it was introduced by Saaty with the aim of evaluating tangible and intangible criteria in relative terms by means of an absolute scale (Saaty, 1980(Saaty, , 2005. The standard process requires firstly the identification of a set of alternatives and a hierarchy of evaluation criteria (value tree), followed by pairwise comparisons to evaluate alternatives' performance on criteria (scoring) and criteria among themselves (weighting) (Belton and Stewart, 2002).
All the weights/alternatives are compared in respect to the criteria by asking the DM his preference on a scale from 1 to 9, with 1 indicating equal preference and 9 absolute preference (Saaty, 1980(Saaty, , 2005. Intermediate values are used to express increasing preference/performance for one weight/alternative. The resulting output of this procedure is a matrix of comparisons expressed as ratios, and the next step is the reduction of the pairwise comparison matrix to a set of scores representing the relative importance of each weight and performance of alternatives (priority vectors) (Belton and Stewart, 2002).
Once the criteria weights and alternatives scores have been derived with the described process, overall performance of the alternative can be calculated by means of a linear additive model (Saaty, 2005). The final result is a value between 0 and 1, where the weights indicate the trade-offs between the criteria (Belton and Stewart, 2002).

Elimination and choice expressing the reality (ELECTRE)
ELECTRE are preference aggregation based methods, working on pair-wise comparisons of the alternatives (Figueira et al., 2005b). They are also defined as outranking approaches because they aim to assess whether option a is at least as good as (in other words it outranks) b (aSb) (Roy, 1996;Belton and Stewart, 2002).
These methods were introduced by Bernard Roy preferences are structured on four elementary binary relations, indifference, preference, weak preference and incomparability (Roy, 1996;Figueira et al., 2013).
In order to identify the outranking relations, concordance and discordance indexes are employed that refer to the cases where the criteria of alternative a are the same or better than those of b (aSb) and to the cases where criteria of a are not as good as those of b (bSa), respectively (Belton and Stewart, 2002;Benoit and Rousseaux, 2003;Figueira et al., 2005a).
ELECTRE methods were developed in order to account for heterogeneous criteria whose aggregation in a common scale is Weakness:no re-evaluation is possible and new software runs need to be performed and independently compared with the previous ones Strength:assessments can be run with new alternatives and compared simultaneously difficult, to prevent compensation behaviour and to account for differences in terms of preferences, leading in this way to the introduction of thresholds (Figueira et al., 2005b). Several ELECTRE methods have been developed to solve different decision problems, including choosing, ranking or sorting, and extensive description of their theoretical and mathematical principles, extensions and applications can be found in (Roy, 1991;Figueira et al., 2005b;Figueira et al., 2013).

Preference ranking organization method for enrichment of evaluations (PROMETHEE)
PROMETHEE methods are also part of the outranking MCDA family and were developed by J.P. Brans during the early eighties, and are based on a set of prerequisites (Brans and Mareschal, 2005): (i) the extent of difference between the performance of two alternatives must be accounted for; (ii) the scales of the criteria are irrelevant as comparisons are performed on a pairwise base; (iii) three cases are possible: alternative a is preferred to alternative b ; alternative a and alternative b are indifferent; alternative a and alternative b are incomparable; (iv) the methods should be easily understandable by the decision makers; and (v) weights must be assigned in a flexible manner.
The assessment procedure requires information between and within the criteria. Concerning the information between the criteria, this is expressed as the relative importance among them and consists of weights that are independent form the measurement scales (Behzadian et al., 2010).
In order to obtain information within the criteria, a preference function for each criterion, expressing the difference in performance of alternative a over alternative b must be identified, adopting as a result the pairwise comparison approach (Belton and Stewart, 2002). Six different shapes for the preference function have been defined, and the identification of the appropriate ones is a task of the analyst who has to question the DM in a structured manner (Brans and Mareschal, 2005).
Once the preference functions for all the criteria and the weights (w i ) of the criteria are identified, a comprehensive preference index indicating the degree of preference of a over b can then be calculated as the weighted average (Behzadian et al., 2010). Subsequently, 2 parameters, leaving and entering outranking flows must be calculated, indicating the outranking power and weaknesses of each alternative over the other, respectively. Lastly, the leaving and entering flows can be combined, resulting in the net outranking flow that provides the performance of each alternative (Brans and Mareschal, 2005).

Dominance-based rough set approach (DRSA)
The dominance-based rough set approach is a relatively new technique which can handle classification, choice and ranking problems and it was introduced by Greco and Slowinski groups (Greco et al., 1997(Greco et al., , 1998(Greco et al., , 1999(Greco et al., , 2001b. DRSA is based on an information table whose rows are defined as alternatives, while the columns are divided into condition attributes; namely the criteria that are needed to assess the alternatives and the decision attribute, which represents an overall evaluation of the alternative. This can be a well-defined concept or an expert judgement (Slowinski et al., 2009).
When a DM is involved in the process he/she is asked to select a class where each alternative belongs or to compare one alternative with the other and decide which one performs better, without the need to specify any weights or thresholds as it was in the previous methods .
DRSA approximates the information reported with the decision attributes by considering the knowledge reported in the condition and decision attributes in the form of "if . . . then . . . " decision rules . These rules are simple connections of elementary conditions between condition and decision criteria, and in the case of classification with criteria g and alternatives a and b , the output has the following syntax : "if g 1 ! r and g 2 ! t and . . . g x ! l, then a is assigned to category p or better", "if g 1 r and g 2 t and . . . g x l, then a is assigned to category p or worse".

Research methodology
The aim of this work was to perform a state-of-the-art review of the MCDA methods described above, investigating their capabilities of supporting sustainability assessments.
The MCDA methods were analyzed in relation to a set of ten comparison criteria that have been indicated as fundamental by many authors when dealing with sustainability-related research (Teghem et al., 1989;Belton and Stewart, 2002;Benoit and Rousseaux, 2003;Munda, 2005;Polatidis et al., 2006;Munda, 2008;Sadok et al., 2008;Buchholz et al., 2009;Antunes et al., 2012;Rowley et al., 2012;Sala et al., 2013a). They have been clustered according to the approach proposed by Bockstaller et al., (2009) in three domains: (i) "scientific soundness", (ii) "feasibility" and (iii) "utility". The list of the ten comparison criteria is reported in Table 1, together with their description and the rationale for the positive or negative assessment of the MCDA methods in relation to each of them.
A literature review was adopted as research methodology to evaluate the MCDA methods with respect to the comparison criteria and the target database was Web of Science (WOS), which includes more than 12,000 journals and 30,000 books worldwide (Reuters, 2014). Furthermore, the Journal of Multi-Criteria Decision Analysis and Integrated Environmental Assessment and Management were searched individually, as they are excluded from WOS.

Results
The results of the comparisons of the MCDA methods based on the ten comparison criteria are shown with the related references in Table 2. Three symbols have been used to indicate the performance (i.e. + = good, indicating strength of the set of methods; = intermediate, indicating a dependence of the specific method within the set or the authors referenced interpretation; À = poor, indicating weakness of the set of methods) of each group of methods in relation to each criterion.

Use of qualitative and quantitative data
All the considered methods can handle information that is qualitative and quantitative in nature, with the qualitative being reduced to point scales (Antunes et al., 2012), with the exception of DRSA, which does not require any data transformation (Greco et al., 2001b). The flexibility on the input side is one of the main upsides of MCDA methods, since it does not pose restrictive requirements on the analyst in terms of problem structuring, as it might be with more data intensive techniques (e.g. life cycle assessment, risk assessment) or in approaches based on optimization.

Life cycle perspective
It can be seen from Table 2 that all the MCDA methods analyzed are able to include a life cycle perspective. The specific literature on MCDA methods does not discuss this aspect in particular, nevertheless, their structure does not limit the number and type of criteria to be used as input parameters (Belton and Stewart, 2002;Figueira et al., 2005a;Slowinski et al., 2009), so that it can be affirmed that all the life stages of a target object can be accounted for.

Weights typology
The main distinction between the weights typology is between coefficients of importance and trade-offs or substitution rates (Munda and Nardo, 2005). MAUT and AHP are based on an additive/multiplicative aggregation model and the weights represent the "gain with respect to one variable allowing to compensate loss with respect to another" (Munda, 2005), in other words they are the trade-offs than can be accepted among the criteria (Belton and Stewart, 2002;Polatidis et al., 2006;Munda, 2008). This has remarkable implications on the aggregation procedure, as it indicates that the scaling of the criteria and the weights are highly linked and dependent, and consequently if one changes, the other has to change accordingly (Belton and Stewart, 2002;Rowley et al., 2012). It must be noted that in the case of the AHP there are authors that consider the weights as importance coefficients (De Montis et al., 2000. On the other hand, importance coefficients indicate the voting power of the criteria; they are expressed with an ordinal meaning and are representative of non-compensatory methods (Belton and Stewart, 2002;Figueira et al., 2005a;Munda, 2005). They indicate the power of the criterion in contributing to the building of the outranking relation, and they are independent from the measurement scale of the criteria (Figueira et al., 2005b). This is considered as undisputable in the case of the ELECTRE methods, while it is not the case for the PROMETHEE ones, whose founders affirm that the weights are not trade-offs and must be used as coefficients of importance (Brans et al., 1986;Le Teno and Mareschal, 1998;Belton and Stewart, 2002;Brans and Mareschal, 2005;Rowley et al., 2012). On the other hand, some researchers indicate that they are rather trade-offs (De Keyser and Peeters, 1996;Munda, 2008).
This criterion is not applicable to DRSA because the approach works without direct weights elicitation from DMs and it extracts this type of information indirectly from the reducts (i.e. sets of criteria which maintain the quality of the approximations as the whole set of criteria) that derive from the classification, choice or ranking assessment (Greco et al., 2001b;Slowinski et al., 2012).

Thresholds values
Thresholds can be used for two main reasons, the first one being that they allow to account for indifference and preference when two alternatives are compared (Mendoza and Martins, 2006) and the second one being that they affect degree of compensation among the different criteria (Buchholz et al., 2009).
For what concerns the basic MAUT and AHP methodologies described above there is no possibility of using thresholds (Buchholz et al., 2009;Antunes et al., 2012). Nonetheless, methods subsequently developed and based on the MAUT methodology allow the inclusion of thresholds (see for example DecideIT software which accepts indifference and also veto thresholds) (Danielson et al., 2007;Buchholz et al., 2009).
The evaluation is highly different for ELECTRE and PROMETHEE, which handle effectively different thresholds as they constitute the  basic structure the methods are based on. ELECTRE require three types, namely indifference, preference and veto, while PROM-ETHEE needs only the first two (Benoit and Rousseaux, 2003;Brans and Mareschal, 2005;Figueira et al., 2005b). DRSA allows identifying thresholds from the decision rules expressed as if and then conditions, so that the if part indicates the threshold values that the criteria must have in order to satisfy a certain assignment, being it a class or a position in a ranking (Slowinski et al., 2009(Slowinski et al., , 2012Roy and Słowi nski, 2013).

Compensation degree
Performance aggregation based MCDA methods (i.e. MAUT and AHP) score badly on this indicator since they assume complete compensation among the criteria and as a consequence they can be used only to enforce a weak sustainability concept (Munda, 2005;Rowley et al., 2012). This is due to the aggregation of all the criteria in a unique value that implies full compensation among them, in other words bad performance on some criteria can be offset by good performance on others (Teghem et al., 1989;De Montis et al., 2000;Benoit andRousseaux, 2003, Munda, 2005;Polatidis et al., 2006;Munda, 2008;Rowley et al., 2012).
On the contrary, preference aggregation based methods (i.e. ELECTRE, PROMETHEE and DRSA) allow the use of a strong concept of sustainability by limiting or abolishing the compensation among sustainability spheres (Teghem et al., 1989;Benoit and Rousseaux, 2003;Munda, 2005Munda, , 2008Roy and Słowi nski, 2013) (Table 2). ELECTRE methods are highly non-compensatory since they consent the introduction of indifference and preference thresholds that limit the compensation and in addition veto thresholds which eliminate alternatives that perform excessively bad in any criteria (Teghem et al., 1989;Belton and Stewart, 2002;Polatidis et al., 2006;Munda, 2008). Higher compensation is reported for PROMETHEE methods, specifically due to the process of obtaining the valued outranking relation dependent on the choice of different preference functions (Teghem et al., 1989;Munda, 2008). Nonetheless, the use of thresholds reduces the degree of compensation (Polatidis et al., 2006).
DRSA is also assessed positively as the rules that constitute the outcome of the assessment represent non-compensatory causal relationships deriving from judgements based on the condition criteria .

Uncertainty treatment/sensitivity analysis
Uncertainty can be accounted for in the case of (i) the criteria weighting and (ii) the performance assessment of the alternatives (i.e. scoring) (Buchholz et al., 2009). Furthermore, a basic distinction is between the treatment of uncertainty at the input stage and at the output one by the support of a sensitivity analysis (Buchholz et al., 2009).
MAUT performs well in this case, in fact it was developed to deal explicitly with uncertain information (Belton and Stewart, 2002). As a result, in the input stage it can manage random and probabilistic input criteria values, and in the case of sensitivity analysis it can cover both the uncertainties of weighting and scoring (Buchholz et al., 2009).
As far as the AHP is concerned, the inconsistency index used is described as an indirect measure of the uncertainty in the criteria weighting step, while at the output stage the sensitivity analysis can only be applied on criteria weights and not on their scoring (Buchholz et al., 2009). Criterium Decision Plus is a software developed to support this methodology and it accepts uncertainties in the form of values distribution, and performs sensitivity analysis on weights (Haerer, 2000;Weistroffer et al., 2005;InfoHarvest, 2014).
ELECTRE and PROMETHEE methods explicitly account for uncertain input criteria scores by the adoption of the pseudo-criterion model that introduces indifference and preference thresholds (Brans and Mareschal, 2005;Figueira et al., 2005b;Polatidis et al., 2006;Rowley et al., 2012).
DRSA is able to handle uncertainty by adopting the concept of stochastic dominance so that probability scores are assigned to the values that the criteria can have (Greco et al., 2001a), or by means of score intervals rather than precise values in the case of imprecise datasets (Dembczy nski et al., 2009).

Robustness
Rankings are considered as robust when the addition or deletion of an alternative does not affect the classification or ranking of all the others. In the case of MAUT this phenomenon cannot take place as each alternative is evaluated independently, whereas this is an issue in all the other methods analyzed here (Munda, 2008;Antunes et al., 2012) (Table 2).
Dyer fiercely criticized AHP affirming that it is a flawed procedure which leads to arbitrary rankings, and proposed its re-alignment with MAUTas a solution (Dyer,1990). On the contrary, Saaty provides a different perspective on the issue, stating that this phenomenon can happen and rather than being a problem, it as a need (Saaty, 1990). In this case, alternatives evaluation is dependent on all the others that are considered, so that the addition of new alternatives or deletion of others determines the restructuring of the decision problem, thus creating a new one (Saaty, 1990).
It has recently been shown that ELECTRE can be affected by rank reversal and this is due to the structure of the decision model which is based on pairwise comparisons and so it is dependent on the overall set of alternatives as is AHP (Wang and Triantaphyllou, 2008). This critique has been questioned by main representatives of the ELECTRE community, who stress the fact that it is hard, if not impossible, to derive the definitive ranking in real-world problems. With a variety of options emerging constantly, as well as criteria, the rank reversal is ascribed to the poor quality of the information available which implies that more input data are needed rather than the method being not reliable (Figueira and Roy, 2009). Figueira and Roy further underlined that the reversal in the rankings is related to the change in the input data (e.g. new alternative) which affects the degree of credibility of the valued graphs and consequently the final rankings (Figueira and Roy, 2009), indicating the understandable and legitimacy nature of this phenomenon. PROMETHEE methods are affected by the same phenomenon as they are also based on pair-wise comparisons, so some explanations reported above are applicable to this MCDA family as well, such as the fact that new and different alternatives modify the pairwise comparisons, in this case the outranking flows calculation. Mareschal and colleagues demonstrated that rank reversal can be restricted to a certain set of cases (Mareschal et al., 2008) and recently the issue has been further investigated (Roland et al., 2012).
Robustness results in DRSA are dependent on the relative support of rules, which means the number of alternatives that follow the rule in relation to the whole number of alternatives in the information table (Slowinski et al., 2009). This implies similar considerations to the AHP, ELECTRE and PROMETHEE cases, and although there is lack of literature about the possible rank reversal issue in DRSA, the fact that the approach is based the outranking relation suggests that DRSA could suffer from rank reversal. Table 3 contains the list of some of the available software that can be used to implement the MCDA analyzed here. DecideIT and DECERNS support MAUT and they allow to visualize alternatives rankings with diagrams and also sensitivity analysis (Buchholz et al., 2009;Linkov and Moberg, 2012).

Software support and graphical representation
DECERNS implements AHP and provides the same graphical potentials as those for MAUT described above (Linkov and Moberg, 2012). Other software package for AHP are super decisions, criterium decision plus and expert choice, which support a good set of results representation, including the partial contribution of each alternative to the total scoring, the evaluation of the effect of different trade-offs, sensitivity and also uncertainty analysis (Haerer, 2000;InfoHarvest, 2014).
ELECTRE methods have different software support on the basis of the type of method adopted. ELECTRE IS, III-IV are freely available (Weistroffer et al., 2005;LAMSADE, 2014) and their graphical representation is low as limited to a diagram representing the ranking or sorting of the considered alternatives (Hokkanen and Salminen, 1997;Augusto et al., 2005;Khalili and Duecker, 2013;Merad et al., 2013).
PROMETHEE methods are the most widely software supported approach in terms of data management and specifically its representation, supporting comparisons of scenarios, visualization of the influence that different weights, criteria, and preference functions can have by means of Decision Lab in the past and nowadays with Visual PROMETHEE and D-Sight (Geldermann and Zhang, 2001;Brans and Mareschal, 2005;De Smet, 2014;Mareschal, 2014).
DRSA is supported by two freely available software developed by the Laboratory of Intelligent Decision Support Systems at the Poznan University of Technology (Slowinski, 2014). JMAF software supports DRSA for classification problems, it is available as a Java application, its interface is user friendly, the results are easily obtained and the manual is accessible for novice users (Błaszczy nski et al., 2013), whereas JRank is a command line Java application (making its use quite challenging for not experts in the programming area) which supports DRSA for choice and ranking problems and the graphical representation is limited .

Ease of use
MAUT application is supported by software (i.e. DecideIT and DECERNS) with simple and intuitive interfaces to structure the assessment and the sensitivity analysis (Buchholz et al., 2009;Linkov and Moberg, 2012). This is the same for the AHP software, whose structure is straightforward and easily understandable (Fernandez, 1996;Buchholz et al., 2009;Linkov and Moberg, 2012;InfoHarvest, 2014).
There are however concerns about the underlying structure of MAUT and AHP as discussed by some researchers (including Keeney, the founder of MAUT), who indicate the high time required by the DMs to assess utility functions or perform pairwise comparisons and the identification of trade-offs between different criteria (e.g. pollution and employment) (Keeney and Wood, 1977;Polatidis et al., 2006;Munda, 2008;Antunes et al., 2012). The latter issue of trade-offs identification is of paramount importance in the additive models, including MAUT and the AHP, as their value is directly related to the adopted measurement scales as already discussed before demonstrating the highly demanding cognitive exercise that the DMs have to make when applying these methods (Keeney and Wood, 1977;Munda, 2008).
On the other hand, although there is not this constrain on the weights, ELECTRE methods score low in this case, primarily due to the high number of parameters to be defined (preference, indifference and veto thresholds), the evaluation procedure based on concordance and discordance indexes, the distillation procedure and the results representation based on the Kernel graphs (De Montis et al., 2005;Polatidis et al., 2006;Munda, 2008).
Concerning the PROMETHEE approach, this is also affected by the time-intensive thresholds identification, but overall it is regarded as easier to understand and use than ELECTRE (Polatidis et al., 2006;Munda, 2008). Furthermore, the software is easy to use, with a very user-friendly interface (Geldermann and Zhang, 2001;Buchholz et al., 2009).
DRSA scores very well in this case as it is characterized by a variety of appropriate features for structuring the decision problem, exploiting it and interpreting the results (Greco et al., 2001b;Slowinski et al., 2009Slowinski et al., , 2012Roy and Słowi nski, 2013): It does not require direct elicitation of cognitive demanding information (weights, thresholds) from the DMs, as it is for other MCDA approaches. The decision model is composed of decision rules expressed in the form of "if . . . , then." conditions, which are transparent and easily understandable by the DMs. The rules are related to specific decision alternatives, which allow tracing and improving the decision process.
Once the decision model is accepted it can be used to support future decisions and easily updated on the basis of new alternatives evaluation.

Learning dimension
DecideIT, DECERNS, super decisions, criterium decision plus, ELECTRE and DRSA software do not permit simultaneous comparisons of the evaluations based on different inputs and, as a result, it is required to re-run the software and obtain independent results (Buchholz et al., 2009;Antunes et al., 2012;Linkov and Moberg, 2012;Błaszczy nski et al., 2013;Szelag et al., 2013;InfoHarvest, 2014).
Expert choice allows comparisons of scenarios with different input so that dynamic reevaluation is supported (Fernandez, 1996).
Decision Lab Visual PROMETHEE and D-Sight offer the widest potentials with the "multi-scenarios analysis" that permits visualization of the various scenarios as 'would-be' alternatives, so that action profiles, walking weights, and multi-representation in the GAIA plane are allowed (Geldermann and Zhang, 2001;De Smet, 2014;Mareschal, 2014).

Discussion
The analysis of the MCDA methods was based on a broad review of literature and not an expert judgement resulting from the studying of the method as some comparative work on indicatorbased method did (van der Werf and Petit, 2002) or on a implementation of the method (Bockstaller et al., 2009).
The review has shown that there is not a clear agreement among different authors concerning some comparison criteria (Table 2). Nonetheless, several considerations can be derived starting from the positive fact that all the methods can conceptually include all the life stages of a target object, being it a product, service or policy.
The use of qualitative and quantitative information in sustainability assessments is fundamental as a wide variety of data typology has to be accounted for, and it was seen that all the methods result to be able to deal with this requirement. However, there are authors who question the explicit inclusion of qualitative or mixed information for the utility and outranking based methods, due to the need of manipulating the information at the input stage (Sadok et al., 2008).
MAUT and AHP can only use a weak sustainability perspective with criteria trade-offs as the norm, whereas ELECTRE, PROM-ETHEE and DRSA enforce a strong one, by limiting or abolishing the compensation among/within sustainability spheres. It must be noted that it is difficult and time consuming to obtain trade-offs from DMs, as they may feel uncomfortable about expressing their compensation acceptances among the criteria, they might not have enough time to dedicate to the lengthy elicitation procedure, or they might simply not have this type of information in mind, suggesting that importance coefficients might be preferred and consequently outranking methods or DRSA (which does not use any weights).
What is more, there has been a misuse of weights in many studies, by eliciting them as importance coefficients when they should have been treated and derived as trade-offs (Belton and Stewart, 2002;Munda 2005;Munda and Nardo, 2005). This is due to the fact that DMs are usually comfortable to express the relative importance of criteria via a semantic scale that indicates ratios among them, but this implies that the information derived represent the importance that they assign to the criteria rather than the trade-offs to be adopted among them (Belton and Stewart, 2002). As a result, if weights want to be used as importance coefficients, it is required to use non-compensatory approaches (Munda and Nardo, 2005). DRSA overcomes this weights elicitation process, reducing consistently the cognitive load for the DMs. Furthermore it extracts the most important criteria indirectly, from the information table, in the form of the reducts.
The use of thresholds has been shown to be an important and useful feature of the MCDA methods since imprecise information can be used and the level of compensation among the criteria can be limited or eliminated. ELECTRE and PROMETHEE perform very well on this as they are inherently based on these features. On the other hand, AHP is not adapted to work with them as the basic MAUT, which affects them negatively in their performance evaluation. However, the identification of exact values for the thresholds has been indicated as a difficult procedure, specifically due to the fact that the DMs could not have these values in their mind (Polatidis et al., 2006). In this regard, DRSA performs better because it supports the identification of thresholds without the need of asking directly the DMs, as it deduces this information from the decisions that were made.
MAUT, ELECTRE, PROMETHEE can consider imprecise data input with probabilistic approach (MAUT) or indifference and preference thresholds (ELECTRE and PROMETHEE), whereas AHP is constrained to sensitivity analysis on weights. DRSA handles uncertain information in a variety of manners, from assigning scores with a certain probability, to defining interval scores, up to handling datasets with missing data.
Software support is provided for all the MCDA methods discussed, although the features of each of them are different and affect the potentials of data communication, analysis and reevaluation. ELECTRE, MAUT and DRSA score worst in this case as no software considered here allows dynamic comparisons of former and successive assessments. AHP, one of the most widely used MCDA methods, is supported by a high number of software with different data management and representation capabilities. More consistency can be found for the PROMETHEE family, whose currently official software, Visual PROMETHEE, provides an extensive set of tools and interactive interface to aid the process of data management. The possibility of dynamic reevaluation can only be achieved by AHP and PROMETHEE, leaving the other methods with a big disadvantage.
Concerning the robustness of the assessments, as the evaluation of each alternative is performed per-se no rank reversal can take place in MAUT. On the other hand, a lot has been published on rank reversal specifically for AHP and some authors see it as a major problem, whereas others consider it as acceptable and legitimate. What characterizes AHP, ELECTRE and PROMETHEE and DRSA is the fact that they are all based on the whole set of considered alternatives. As a consequence, there is no "right" or pre-defined classification or ranking to be identified, but the decision process is rather based on the situation at stake. When the alternatives to be evaluated are changed, this affects the relative scoring in the AHP, the credibility degree in ELECTRE, the outranking flows in PROMETHEE and the dominance relations in DRSA, thus resulting in a new decision situation that cannot be considered as the same as the previous one.
The easiness of use is a fundamental aspect in the MCDA process, specifically when the DMs are not experts in the field. For what concerns MAUT and AHP, a discrepancy has been identified in the literature between the lengthy and cognitive demanding activities that the MAUT and AHP procedures require, in contrast with the easy-to-use and straightforward software have been developed to support them. ELECTRE scores low, since they are the most sophisticated class of methods and require several parameters to be identified, some of which do not have a clear and practical meaning, and the exploitation procedures is perceived as somehow obscure by many authors. The limited graphical potentials aggravate the evaluation even more. PROMETHEE shows a good balance between theory and implementation, whose structure is based on the outranking approach but is easier than ELECTRE and the software is simple to understand, but also very powerful in terms of results representations, adding a lot to the decision making process and re-evaluation. The method that scores the best is DRSA which has outstanding features compared to the other methods so that it has been named a "glass box" due to its transparency and intelligibility . In fact it does not require as mental demanding efforts from the DMs as it is for the other methods and the output is composed of easily understandable and traceable decision rules expressed in the form of "if . . . , then." conditions.

Conclusions
Multi criteria decision analysis (MCDA) is a set of methods that can be used to support the process of decision making by taking into consideration multiple criteria in a flexible manner, by means of a structured and intelligible framework.
MCDA has been put forward as an excellent candidate to perform sustainability assessment, and a variety of applications have emerged. Nonetheless, in the majority of the available assessments, the selection of MCDA method is dependent on the familiarity and affinity with the approach rather than on the decision making situation under consideration.
This review reported and discussed the results of a comparative analysis of five MCDA methods (MAUT, AHP, PROMETHEE, ELECTRE and DRSA) with specific reference to ten criteria crucial for tools aimed at sustainability-related evaluations. The analysis has shown that all the methods can manage mixed data and support a of life cycle perspective.
ELECTRE, PROMETHEE and DRSA score better than MAUT and AHP in terms of enforcement of a strong sustainability approach together with thresholds management. ELECTRE, PROMETHEE and DRSA limit heavily or abolish the compensation degree among sustainability criteria and spheres. A remarkable note is that weights must be used as importance coefficients and not as tradeoffs when strong sustainability is chosen as the driving paradigm.
Sustainability assessments are multi criteria based evaluations, which necessitate the inclusion of a wide variety of data typology with various certainty degrees. MAUT, ELECTRE, PROMETHEE and DRSA can handle uncertain information very well by means of probability distributions and thresholds. Furthermore, sensitivity analysis can be conducted by all the approaches to consider the variability of the results depending on the input values, a needed characteristic in the highly uncertain field of sustainability.
AHP, PROMETHEE, ELECTRE and DRSA can suffer from rank reversal, which has caused several debates about the interpretation and management of this phenomenon. On the whole, this is an issue to be handled with care in the evolving area of sustainability where new information and alternatives become continuously available and need to be included in the assessments. This aspect is linked with the desirable feature of supporting alternatives comparisons between assessments at different stages of information (dynamic reevaluation), which is currently widely supported by all PROMETHEE software and only by one for the AHP (expert choice), leaving the other software and approaches with a big disadvantage.
Overall, the review highlighted the wide potentials of MCDA in supporting an emerging and heterogeneous area as sustainability assessment. The selection of a certain MCDA method has to be based on an appropriate knowledge of the basics of the approach and the evaluation to be performed as well. This implies the recognition that some aspects can be covered only by certain methods and not by others, so that the adoption of the approach is tailored to the decision making situation at stake and not viceversa.