Nearest solution to references method for multicriteria decision-making problems

. by the authors; licensee Growing Science, Canada 1 2 20 ©

In MCDM problems, the decision maker is often ready to adopt the closest solution to the reference values in a choice or ranking problem. The reference values represent the desired results as established subjectively by the decision maker or determined through various scientific tools. In a criterion, the reference value could be the maximum value, the minimum value, or a specific value or range. Also, the acceptances degrees of ranges outside the reference may differ from each other in a criterion. Furthermore, measurements in a criterion may have been obtained with any of the nominal, ordinal, interval, and ratio scales. For the decision problems, including qualitative criteria, the solution cannot be achieved without scaling of criteria with the existing MCDM methods. The purpose of this study is to propose the Nearest Solution to References (REF) Method, a novel reference-based MCDM method, for the solution of decision problems having mixed data structure where references can be determined for criteria. .

Introduction
Multi-criteria decision-making methods have a wide range of applications from simple decision problems encountered in daily life to huge decision problems of multinational companies. Multi-criteria decision-making methods are frequently used especially in the fields of energy, environment, business, industry, health, sustainability and engineering (Hwang & Yoon, 1981;Pirdashti et al., 2011). These methods share the common fact that they are developed based on the structures of specific problems. However, structural differences arising from the components of decision problems make it impossible to obtain effective solutions to all decision problems using the same multi-criteria decision-making method. Decision criteria are the important components that cause this structural difference. The conditions or facts used as the standard used in the solution of decision problems are called criteria. Consequently, if a decision problem uses a single criterion, then it is a question of a single criteria decision-making, and if it uses more than one criterion, then it is called as multi-criteria decision-making (MCDM) (Aktaş et al., 2015). There are different perspectives considered as determinants in the classification of MCDM methods. When the literature is examined, classifications are seen to be made in terms of the compensability of the criteria, the nature or form of the decision problem (classification, selection, sorting, elimination, definition, designing), etc. The most common of these approaches is to divide MCDM methods into multi-attribute decision making (MADM) methods and multiobjective decision making (MODM) methods with regard to the concepts (Hwang & Yoon, 1981). These distinctions do not have definite limits. In other words, it is possible to evaluate a method under more than one class. The most important distinguishing feature of MODM is that alternatives are infinite and not fully known in advance. The main aim of MODM methods is to obtain possible solution alternatives that best meet the constraints at the cardinal measurement level. Thus, the best alternatives are designed using MODM methods, which make use of mathematical programming tools. Therefore, the number of alternatives to satisfy these constraints may be unlimited. The common features of MODM methods are that they have a set of measurable objectives and a well-defined set of constraints as well as the ability to enable the acquisition of some explanatory information between the specified objectives (Hwang & Masud, 1979). MODM methods used in design problems can be divided into three groups as Goal Programming (GP), Linear Programming (LP), and Nonlinear Programming (NP) (Colson & De Bruyn, 1989;Avriel, 2003).
MADM methods are used to obtain solutions for problems of choice, ranking, sorting, elimination, or definition. The most distinctive feature of these methods is that alternatives are obvious and often limited. In MADM, alternatives are not always required to have a numerical measure. In other words, the comparison or evaluations of the alternatives is made according to the performance values within and between attributes. In addition, comparisons may include explicit or indirect tradeoffs (Hwang & Masud, 1979). MADM methods may be classified in different ways. For instance, they may be classified based on compensability (compensatory, non-compensatory, and partially compensatory), the nature of the problem (choice, ranking, sorting, elimination and definition) and the number of decision makers (individual, group) among others. In the extant literature, accepted classification MADM methods are generally divided into three main classes based on the aggregation tools used and how the preferences are obtained. The classifications include Utility-Value Theory-Based Methods, Outranking-Based Methods and Interactive Methods classes. In addition to these, another common classification approach add Elementary Methods, Fuzzy Set Theory, Definitive Methods (Principal Component Analysis, Multidimensional Scaling, etc.) and Reference (Goal, Compromise Solution) Based Methods categories. Some MODM methods are also used in fuzzy and interactive applications. The nature of the classification of the methods notwithstanding, some methods can be evaluated in more than one category (Colson & De Bruyn, 1989;Stewart, 1992;Belton & Stewart, 2002;Ramanathan et al., 2017). The concepts of utility and value have come to play an important role place in decision theory. While the concept of value is to some extent a transformation of physical measurement, utility incorporates the decision makers attitude towards uncertainty and risk. These concepts are considered as the value or utility expected by the decision maker from the solution of the decision problem and have led to the development of many theories, especially Expected Utility Theory and Prospect Theory (Von Neumann & Morgenstern, 1953;Kahneman & Tversky, 1979;Keeney & Raiffa, 1993;Gass & Fu, 2013). These theories are mainly one-dimensional. Keeney and Raiffa (1993) stated that reallife problems involve many criteria and are therefore multidimensional. They determined that with their proposed Multi-Attribute Utility Theory and utility function, the decision maker can reach the solution of the decision problem by going through trade-offs between criteria. When the solution results of the problem are certain, the utility function is defined as the value function. While choosing the alternative with the best value using the value function, the attitude of decision maker towards risk is taken into account in the utility function. The rules used in problem-solving with the value function are collectively known as Deterministic Utility Theory (Chankong & Haimes, 1983;Von Winterfeldt & Edwards, 1986;Colson & De Bruyn, 1989;Stewart, 1992;Keeney & Raiffa, 1993).
Methods based on Multi-Attribute Utility/Value Theory are also referred to in the literature as American School Decision Models (Colson & De Bruyn, 1989). The common thread of all these methods is that they evaluate the alternatives based on the criteria and aggregate these evaluations to create a unique evaluation result for each alternative. In these evaluations, the utility or values of alternatives with regard to the criteria are assumed to be independent of each other and that the sum of these utility or values represents the total utility/value. MAUT (Multi-Attribute Utility Theory), MAVT (Multi-Attribute Value Theory) and SMART (Simple Multi-Attribute Rating Method) are some of the methods that can be exemplified under this topic (Keeney & Raiffa, 1993;Colson & De Bruyn, 1989;Ramanathan et al., 2017). Outranking-based methods are based on the idea of defining the superiority relationships between alternatives. These methods are based on the view that it is more accurate to compare the alternatives in each criterion based on their superiority rather than the measurability of the utility. These methods are also referred to in the literature as French / European School Decision Models (Colson & De Bruyn, 1989). In outranking-based methods, an expression like A> B in the binary comparison of two alternatives, A and B, indicates that A is superior to B within the relevant criterion. On the other hand, there may be an indifference, or incommensurability between the two alternatives, or the superiority may be in the second alternative (B> A). The basic condition for the superiority of alternatives to each other is that in case A> B, A is an absolute superior to B, and in case of A≥ B, A is at least as good as B. The most known outranking-based multi-criteria decision-making methods are ELECTRE (ELimination Et Choix Traduisant la REalité) and PROMETHEE (Preference Ranking Organization Method for Enrichment Evaluations) (Colson & De Bruyn, 1989;Belton & Stewart, 2002;Greco et al., 2016). Interactive methods enable the gradual optimization of preferences by interacting with the decision maker during the solution of the decision problem. There are two important actors in interactive methods: the analyst and the decision maker. In these methods, a feasible and effective solution is obtained by the analyst in a way that reflects the decision maker's preferences. This solution is then presented to the decision maker for feedback. If the decision maker does not find the solution sufficient, a new (better) solution is sought and forwarded to the decision maker. This process is repeated until the decision maker is satisfied with the solution or a certain stopping criterion is met (Miettinen, 1998;Ramanathan et al., 2017). The most important assumption of interactive methods is that the decision maker has sufficient level of knowledge and evaluation time. The learning time of the decision maker, his desire to take part in the whole process and make no repetition in their preferences are also important. In this context, it is difficult to obtain effective results with interactive methods in complex problems, especially in the field of engineering. STEM (the STEp Method) and the Reference Point Method are some examples of interactive methods (Miettinen, 1998). As mentioned earlier, some methods may be classified in more than one category. Reference-based methods can, for instance, be evaluated under both utility-value-based and interactive method. The reference, in the reference-based methods, is the application source and guide that reflects the decision maker's preferences in the decision problem. In the multicriteria decision problem, the reference values determined in the criteria are used to evaluate the alternatives. The reference has a broad meaning, including ideal and utopian concepts. In this context, in the solution of multicriteria decision problem, the reference, which may be an ideal or utopian value/interval, is the basis of comparison. In a choice or ranking problem, the decision maker is often willing to adopt the closest alternative to the reference values that he has set based on the criteria. The compromise solution obtained from the reference values is defined as the solution that optimizes many purposes at the same time in the decision problem. This definition shows the basis of reference based methods (Yu, 1973;Zeleny, 1974).
In reference-based methods, the decision maker determines any point or range of values from the solution space. Thus, the decision maker regulates the solution set according to these references. The closest solution compatible with the references is then determined using distance measurements. These reference values represent the desired results that are determined subjectively by the decision maker or determined by various scientific tools. Examples of reference-based methods that have an important place in the literature are TOPSIS, VIKOR, EDAS, CODAS, RIM and CP (Yu, 1973;Zeleny, 1974;Opricovic & Tzeng, 2004;Kaliszewski, 2006;Wachowicz, Brzostowski & Roszkowska, 2012;Pomerol & Barbara-Romero, 2012;Keshavarz-Ghorabaee et al. 2015;Cables, Lamata & Verdegay, 2016;Keshavarz-Ghorabaee et al. 2016). In reference-based methods, the reference value is mostly set as the max / min value (point) based on the values in the decision matrix and the optimization aspect. In real life problems, the reference in a criterion could be the maximum value, the minimum value, or a specific value or a certain range. The acceptance degree of values/ranges outside of the references may differ from each other. The measurement levels of the criteria may also be at the nominal, ordinal, interval, ratio scales. For aforementioned decision problems, the existing MCDM methods cannot provide solutions without scaling of criteria. In other words, in the majority of existing MCDM methods, the qualitative criteria are re-measured using different scales (pairwise comparison, Likert type scale etc.) and converted at least to the interval scale level, and only then can the solution be finished. The purpose of this study was to propose a reference based MCDM method for the solution of decision problems in a specified class. The study proposes the Nearest Solution to REFerences Method (REF).

REF-Nearest Solution to References Method
REF grants the decision maker freedom without restricting him to a certain range, pattern or scale in their preferences. In REF, the criteria measured either by nominal (binary and multinomial), ordinal, interval or ratio scales can be used together, and the decision maker can determine a certain point, range or category from these criteria as the reference. REF allows the decision maker to determine the desired results independently of the decision matrix based on the characteristics of the criteria. Thus, REF aims to solve the decision problem by understanding the aspirations and goals of the decision maker in the best manner. The decision maker takes into consideration many criteria when ranking or choosing alternatives in the decision problems. With REF, the aim is to determine the distance of the alternatives from the reference points/ranges with the appropriate distance measure, then aggregate the obtained distance values using an aggregation tool, and finally, rank the alternatives in ascending order based on these global values. Before proceeding to the implementation steps of REF, some concepts need to be explained in the context of the decision matrix X in Eq. (1).
In Eq. (1), the performance value of alternative i in criterion j is represented by xij. The reference is a specific value or range of values that the decision maker uses to evaluate the alternatives in the context of decision criterion j. If the reference is given as a specific value for criterion j, this value is expressed as . However, if the reference ( ) is specified as a range, the lower limit of this range is expressed as , and the upper limit is expressed as . The decision maker may have the view that the distances from the reference point should be evaluated differently at certain range/point, or in a particular direction in the decision problem. Values to the right of a reference point/range may seem more acceptable than values to the left, or the preferability levels of values on the right and left of different ranges may differ from each other. In this case, the value/range considered to be less important than the reference is called the successor value/range. The successor value/range of criterion j is denoted by . Accordingly, in criterion j, it is possible to determine as many successor ranges/values as v = 1,…, q. The decision maker determines the effect of successor ranges on the decision problem symmetrically or asymmetrically. For this purpose, an unacceptance value ( ) is assigned to each successor value/range. reflects the unacceptable level of the value remaining within the successor range relative to the reference point/range. The assigned by the decision maker takes a value between 0-1. The decision maker should consistently determine of the respective successor values/ranges for the solution. Scientific tools or techniques can be used to determine . To show multiple ranges outside the reference and the values for these ranges, Fig. 1 was created.  Fig. 1, the decision maker has determined as the reference range for criterion j. The lower limit of the range is shown by and the upper limit by . The distance of alternatives in this range from the reference value in the relevant criterion is zero. The decision maker has different options on the preferability levels of values outside this reference range. For this reason, he aims to reach a more effective solution by determining more than one range. In this context, the solution of the decision problem is reached using the unacceptance value for each range. In Fig. 1, while the unacceptance value ( ) of the range is zero for criterion j, the unacceptance values of the other ranges are shown by , where = 1, … ,6. At this point, values can be assigned as reference or successor values for each criterion j, independent of the decision matrix. Ranges can then be defined using these values as in Eqs. (2-8) in Fig. 1 The decision maker can determine as many successor values/ranges as he deems necessary for the solution of the problem.
However, values in a particular direction from the reference may be considered more acceptable than those in the other direction. For this purpose, the unacceptance value is used again. When this value is set to 0, the distance of the relevant direction to the reference value equals zero, while the effect of the distance increases as the value approaches 1. For instance, the successor neighborhoods of the reference value are shown in Figure 2.

Fig. 2. Successor neighborhoods of the reference value in REF
In Fig. 2, the decision maker determined Rj as the reference point. The left side of this point extends to -∞ and the right side extends to +∞. The solution of the problem can be completed by defining the unacceptance values as 1βj for 1Vj on the left side of RJ and as 2βj for 2Vj on the right side of RJ. After making the necessary definitions of the REF, the application steps of the method are given in the following section.
Step 1: Defining the decision problem and generating the first decision matrix In this stage, the decision problem and the criteria to be followed in its solution are determined. The alternative set for the solution is then determined. The decision matrix X is created with criteria j = 1, 2,…, n and alternatives i = 1,2,…, m as specified in Eq. (1). In this matrix, xij's shows the features or performance value of alternative i in criteria j.
Step 2: Determination of reference and successor points/ranges and unacceptance values for the criteria The decision maker determines the reference points/ranges in the identified criteria. He then determines the unacceptance values for criteria containing successor points/ranges. The decision maker may determine the reference point as a specific category or as the most/least observed in qualitative criteria.
Step 3: Obtaining distance values The appropriate distance measure is used by looking at the scale of measurement of the criteria and the reference points/ranges determined by the decision maker. In this context, the measurements (dij) to be used to calculate the distance of the alternatives from the reference points (Rj) according to the measurement level of the criteria in decision matrix X are given in the following section.
In qualitative criteria, the decision maker can determine the reference as a specific category or the frequency of observation of the categorical values. If a certain category is accepted as a reference in criteria measured by the nominal scale, the distance of the alternatives from the reference point is obtained through inconsistency. For example, the distance of the alternative i from the reference point in the qualitative criterion j is determined as 0 if the point is the same as the reference point, and 1 if it is not. In this context, the distance of the alternative i from the reference point in criterion j is specified in Eq. (9).
The distance value (dij) is obtained using Eq. (11), where the reference point of the qualitative criterion j is .
In criteria measured using ordinal scale, the reference point is determined on the specific rank. In this case, the distance of alternative i from the reference in criterion j, where is the number of ranks in criterion j, is obtained as shown in Eq. (12).
In Eq. (12), the unacceptance value of the interval/point v, is shown by where = 1, … , . In the criteria measured using the ordinal scale, if the successor point/range isn't used, the unacceptance value cannot included in the calculations. On the other hand, if the reference range is specified, Eq. (13) is used.
The decision maker may scale the qualitative criteria as deemed necessary. In this case, the distance measure used for the criteria measured with at least interval scale level are used in the relevant criteria. If the reference is determined as a single value in cardinal (measured with interval or ratio scale) criteria, the distances are obtained using Eq. (14).
On the other hand, if the reference is specified as a range, Eq. (15) is used.
After calculating the distance (dij) of the alternatives from the reference point/range, the decision matrix D consisting of these distance values is obtained using the Equation (9-15).
Step 4: Performing the normalization process Normalization is used to make the criteria comparable by eliminating the optimization direction (benefit, cost), unit of measure and variation range (Yoon & Kim, 1989;Pavlivic, 2001;Jahan & Edwards, 2015;Poviezko & Podvezko, 2015). In this method, Linear Sum-Based Normalization is used due to all distances measure depending on L1 metric. The normalized values are obtained as shown in Eq. (16).
= ∑ Also, this normalization process will ensure that the performance values (fij) of alternatives are between 0-1 and will significantly reduce any criteria from suppressing other criteria due to the width of its range.
Step 5: Deciding on the weights of the criteria If the decision maker believes that the criteria have different levels of importance in MCDM problems, this problem can be eliminated to some extent by weighting the criteria (Choo et al., 1999;Zardari et al., 2015). After the normalization, the values are multiplied by the criterion weights, and the normalized weighted distance values are found. For this purpose, Eq. (17) is used.
Step 6: Obtaining general performance values The overall performance value (Ui) for each of the alternatives is obtained using Eq. (18) Step 7: Ranking the alternatives The solution of the decision problem is completed by ranking the alternatives in ascending order according to the general performance value (Ui).
REF provides the decision maker with the opportunity to set the reference as a single point or range, and to weigh the distance of different ranges or points from the reference. The flexibilities on the criteria makes it easy for the decision maker reach the solution in line with his goals and objectives. These features of the method enable it to be distinguished from other MCDM methods in the extant literature.

Application
In this section, the applicability of REF will be explained in detail with an illustrative decision problem. This is followed by the solution of a real-world problem using REF.

Illustrative example
The illustrative example used to demonstrate the applicability of REF has four criteria and six alternatives. Since the criteria are measured using different scales like ratio, ordinal, nominal (binary and multinomial), the problem has a mixed data structure. The decision matrix in equation (1) and other information related to the problem are given in Table 1. The criteria in Table 1 were measured using ratio, ordinal and nominal scales. Four successor ranges were assigned to C1 and one successor point to C3, and these were then paired with the associated unacceptance values. In C2, with a multinomial data structure, the reference category was determined as the most frequently observed category. C4 is binary structured and the reference category in this criterion is determined by finding the relevant feature (Yes). To demonstrate the implementation of the REF, the calculations for A1 are presented. Since the reference in C1 has been given as a reference range and the performance value of A1 is between the upper range of the reference and the first successor range, the calculation is done as follows.
= |65,5 − 65| = 0,5 C2 was measured using a nominal scale and the reference was determined as the most frequently observed category. Equation (10) calculates the relative frequency value of each categorical value. Since A1 has the most frequently observed categorical value, the distance value (d12) will be equal to 0 as seen below.
C3 was measured using ordinal scale and the rank values were coded as 1 to 5, from the worst to the best. The reference point is 5. In this context, the distance of A1 from the reference point (d13) will be determined as follows: C4 was measured using the nominal scale and has binary data structure. Since the reference in this criterion is determined as the presence of a certain categorical value, the distance (d14) will be calculated with the reference category using the distance Eq. (9).
After applying the equations (9-16) in the second part of the study, the normalized decision matrix is created as in Table 2. After weighting the normalized decision matrix with the criteria weights determined by the decision maker, the weighted normalized decision matrix is obtained as shown in Table 3. The results in Table 3 shows that alternative A4 is ranked first. The performance values of A4 are found within the reference points/ranges of three of the four criteria. This feature was influential in ranking the alternative in the first place in the solution of the problem. The complete order of the alternatives is A4> A2> A5> A1> A3> A6.

A real-world application
This part of the study considered the decision problem in the choice of used cars within a given date range and a certain location. The use of cars has spread very quickly in the 20th century to become an important part of our lives. The car, which is still an integral element of modern life today, has been defined as a four-wheeled land vehicle with an engine 1 . Although cars have gradually become more accessible throughout history, there are still many obstacles for individuals to get the cars they want, the main one being the price. With respect to Turkey, the prices of cars are seen to be higher than the purchasing power of the average citizen. This is attributed to the lack of a domestic car brands and high taxes. On the other hand, the high prices of new cars have led to the effective rise of the second-hand car market, which in turn has made cars more accessible to most people (Güneş, 2012). The sale of previously used cars to other people by owners or intermediaries in markets, auto galleries or e-commerce marketplaces is known as second-hand car/automobile sales. The few studies that exist on the second-hand automobile market in Turkey are mostly aimed at determining the variables affecting price as well as making price estimates of the second-hand automobiles (Asilkan & Irmak, 2009;Daştan, 2016). The decision problem aims to help in choosing the most suitable vehicle in the SUV category under the age of five years in Eskişehir based on the notices of sale published between 1-31 January 2020 in a nationally recognized online marketplace that provides consumer-to-consumer and business-to-consumer sales 2,3 . In solving this decision problem, firstly, the first decision matrix specified in Step 1 of REF is created according to equation (1). In the second-hand automobile choice problem, it is necessary to evaluate many alternatives under several criteria. The criteria to be used in solving the problem and the reference points related to them are presented in Table 4 according to Step 2 of REF. In Table 4, the weight values of the criteria are subjectively determined by the decision maker. 1-5 type Direct Rating Method was used to do this (Zardari et. al 2015). The decision maker has determined the K1 and K10 as the most important criteria. While determining the reference points related to the criteria, if the reference is a single point, the reference value is written to both the lower limit and the upper limit. Besides, the problem has criteria measured using different scales, such as nominal (binary and multinomial), ordinal, interval and ratio. Table 4 shows K11, K15 have binary data structure. K8 and K4, which are among the criteria measured using the nominal scale, the most frequently observed category was taken as the reference category. K12, K13 and K14 were measured using the ordinal scale. In coding the categories in these criteria, the categories were coded by assigning the total number of the categories (sj) as the value of the first category and going all the way to 1, from the best to the worst. The decision maker may decide that points other than the reference point in some criteria present different levels of effectiveness in solving the problem. He can express this view using different points and/or ranges of unacceptance. In Step 2 of REF, the criteria assigned for successor points/ranges and unacceptance values in the second-hand automobile selection problem are presented in Table 5.  Step 7 of REF, the solution of the problem is completed by ranking the U i values for alternatives in ascending order. The rankings obtained are given in Table  6. An examination of the results shows that A13 is ranked in the first place. Accordingly, the decision maker will choose A13 which satisfied reference points/range in nine of the sixteen criteria. In general, REF can be said to have been successfully applied in the second-hand automobile choice problem with criteria measured with different scales and where the references and successor are determined as point or range.

Table 6
The Weighted Normalized Decision Matrix and Ranking Results for Second-hand SUV Choice Problem

Comparative Sensitivity Analysis
Sensitivity analysis is the examination of the decision problem solution with parameter or model inputs changes. Sensitivity analysis is a basic tool in the implementation and effective use of quantitative decision methods. With the sensitivity analysis, the decision maker has the chance to see which parameter, data and component in the decision problem is effective or critical to the solution. Thus, he has the opportunity to change his preferences or judgments in the solution of the problem (Triantaphyllou, 2000;Delgado & Sendra, 2004). There are many sensitivity analysis applications in decision models created in Operations Research and Management Science. However, the number of studies on sensitivity analysis for decision models created under MCDM remains limited. On the other hand, the studies in the MCDM literature were taken as a basis in carrying of the sensitivity analysis for REF, and comparative sensitivity analysis is adopted with prominent reference based methods (Gomes & Rangel, 2009;Keshavarz Ghorabaee et al., 2015;Cables et al., 2016;Stevic et al., 2019). Therefore, this study adopted one of the decision problem solution approaches used in the literature to perform the sensitivity analysis. The rental house evaluation problem, which had been solved using TODIM (Tomada de Decisão Iterativa Multicritério; Interaction and Multi-Criteria Decision Making) method by Gomes and Rangel (2009), was preferred because it contains criteria with qualitative data structure. On the other hand, the purpose of this study is to perform a comparative sensitivity analysis. Accordingly, the initial decision matrix and some information regarding the problem is given in Table 7.  A1  3  290  3  3  1  6  4  0  A2  4  180  2  2  1  4  2  0  A3  3  347  1  2  2  5  1  0  A4  3  124  2  3  2  5  4  0  A5  5  360  3  4  4  9  1  1  A6  2  89  2  3  1  5  1  0  A7  1  85  1  1  1  4  0  1  A8  5  80  2  3  1  6  0  1  A9  2  121  2  3  0  6  0  0  A10  2  120  1  3  1  5  1  0  A11  4  280  2  2  2  7  3  1  A12  1  90  1  1  1  5  2  0  A13  2  160  3  3  2  6  1  1  A14  3  320  3  3  2  8  2  1  A15  4  180  2  4  1  The comparative sensitivity of REF and reference-based methods such as TOPSIS, VIKOR, EDAS, CODAS, and RIM with TODIM will be generated with regards to criteria weights' change, adding and removing criterion, the presence of qualitative criteria in the problem, adding new alternatives which are better or worse than the current ones in Table 7, and parameter value's change (Triantaphyllou, 2000;Delgado & Sendra, 2004;Keshavarz Ghorabaee et al., 2015;Stevic et al., 2019). The solutions obtained in the sets created within the scope of the comparative sensitivity analysis are presented in Appendix-A. Among the criteria in Table 7, K1, K3, K4 were measured using the ordinal scale while K8 was measured using the nominal (binary) scale. In the coding regarding the location of the houses in K1; 1 shows the location on the periphery, 2 shows the location between the periphery and average location, 3 the location on the average location, 4 shows the location on the good location, and 5 shows the location on the perfect location. K3 shows the construction quality of the building, and here, 1 means a low standard, 2 shows average standard and 3 is high standard. K4 considers the state of conservation of the house with respect to the need for renovation. Here, 1 implies the house is in a bad state, 2 is average, 3 is good and 4 shows very good conservation. K8 has the presence of security with 0 meaning no security and 1 signifying the presence of security. K5 and K7 consider the number of specific properties of the house and hence are measured with the ratio scale (Gomes & Rangel, 2009). N-way or one-way approaches are used to evaluate the effects of criterion weights in sensitivity analysis (Chambal et al., 2011). In this study, objective weighting methods such as Equal Weighting, CRITIC, Entropy, standard deviation and variance procedure are used in the application of n-way sensitivity analysis (Zardari et al., 2015). The decision matrix was normalized and used in the determination of weight values with the standard deviation and variance procedure. Thus, apart from the original decision set, five sets were created, which would allow evaluation based on five different criteria weight values.
A sensitivity analysis where one criterion weight is kept as the incumbent while changing the weight values of some criteria and monitoring these processes on the solution of the problem is known one-way sensitivity analysis (Chambal et al., 2011). In this study, Customizable One-Way Sensitivity Analysis (COSA) developed by Chambal et al. (2011) was used. The application steps of COSA are outlined in the following section. In COSA, N represents the set of all criteria to be selected in the sensitivity analysis, including the incumbent, I represents only the set of criteria whose weight value will be changed in the sensitivity analysis, and U represents only the set of criteria whose weight value will not be changed in sensitivity analysis. The change in the weight value of the incumbent criterion ( ) is determined using the equation (19), ws being the weighting of the incumbent criterion.
In equation (19), − shows the maximum weight value of incumbent criterion that can be reduced while the right side of equation shows the maximum weight value of incumbent criterion that can be increased. Since the criterion weights are normalized and take values in the range of 0-1 in accordance with the approach adopted in multi-criteria decision-making, the weight of the incumbent criterion will be between 0 and 1. The upper and lower limit values in equation (19) are limited accordingly. In equation (19), while is the weight coefficient of elasticity for criterion i, is the first (original) weight value of criterion i. The decision maker can determine the value proportionally or he could determine the value by himself provided that it is greater than 0. To use COSA in the sensitivity analysis, criterion K1 which has the highest weight value in set 1 (which is in the original form of the decision problem) was chosen as the incumbent criterion. The six sets (Set 7-12) were obtained by putting K2 and K4, which have highest weight value after K1, in set I and the others in set U. It was also deemed appropriate to share the weight value that will increase/decrease the incumbent criterion equally between K2 and K4. Accordingly, The change in the weight value of the incumbent criterion is −0.075 ≤ ≤ 0.075, as the weight value of the incumbent criterion is 0.175 ≤ ≤ 0.325. In this context, solutions were obtained using w1= 0.175 in set 7, w1= 0.20 in set 8, w1= 0.225 in set 9, w1= 0.275 in set 10, w1= 0.30 in set 11 and w1= 0.325 in set 12. Accordingly, the weight values of the criteria, including the original set (Set 1) are given in Table 8. The new weight values of the twelve sets created based on the change of weight values of the criteria are shown in Table 8. Sets 7-12 were created using the weight values expressed in Set 1, which is the original version of the problem. Sets 2-6, on the other hand, were obtained using the decision matrix in Table 7. Accordingly, K1 has the highest weight value in Sets 1,9,10,11 and 12,K8 in Sets 4,5 and 6,K4 in Sets 7 and 8,and K7 in Set 3. In Set 2, the weights of all criteria are equal. The solutions were for Set 1-Set 12 using REF and TODIM, RIM, TOPSIS, VIKOR, EDAS, CODAS. As can be understood from the original form of the problem, K1, K3, K4 were measured using ordinal scale while K8 was measured using nominal (binary) scale. In this context, REF was used to obtain two different solutions by considering the situations in which the criteria to have been measured with, at least, the interval scale and also evaluated with their actual scale. The other methods cannot be used since they do not propose solutions for criteria measured by nominal and ordinal scales. For optimization aspect of the criteria, it was adhered to the original problem while obtaining solutions. In this context, while the maximum values in decision matrix were taken as the references/ideal points in all criteria, the minimum values in decision matrix were determined as anti/negative ideal in methods that need anti-ideal points. When the solutions obtained with REF under the assumption that all criteria have quantitative data structure are examined, A5 took the first place in all sets. A5 also took the first place in all sets in the solutions obtained with REF where K1, K3, K4 were measured using the ordinal and K8 was measured using the nominal scale. In both scenarios, the reference points were determined by adhering to the original form of the problem and no successor point/range and unacceptance values were assigned. Consistency with the reference, and not the frequencies, was taken into consideration in the K8. The correlation of rankings between sets was examined using the Spearman rank correlation coefficient. The statistical significance of the correlation coefficient indicates the strength of the relationship between the two variables. Accordingly, a correlation coefficient in the range of 0-0.49 shows a weak relationship between variables while a correlation coefficient of 0.5-0.69 shows a moderate relationship between variables and any correlation coefficient more than 0.70 shows a strong relationship between variables (Sümbüloğlu and Akdağ 2007). Considering Set 1-Set 12, REF has very strong rank correlation (rs ≥ 0.9) with TODIM, TOPSIS, VIKOR, RIM, EDAS and CODAS. In the sensitivity analysis performed on the change of the criteria weights, it was observed that there was no major change in the results in terms of the choice problem in the sets except for Set 4. REF can be said to respond similarly to other methods in these sets. It has thus been demonstrated that REF can also be used in the decision problems in which other methods can be used. While A5 ranked first in 87.12% of the solutions generated in twelve sets with different decision making methods, the most change was observed in Set 4. The effects of removing criteria and adding new criteria are examined in the criteria set change. In this context, as seen in Table 4, K1 with the highest weight value, K5 with the least variability and K2 with the highest variability in the criteria set, was removed individually from the decision problem in each process. Thus, Set 13, Set 14 and Set 15 were obtained. On the other hand, the price criterion was created using the solution obtained by Gomes and Rangel (2009), and added to the problem as a new criterion (K9), so Set 16 was created.
The priority values used in the article of Gomes and Rangel (2009) have been used in the reconstruction of the criteria weights. 4 importance values are assigned to K9. To replace the price criterion in Set 16, criterion K9, which has 0, positive and negative values was generated in Set 17, and a criterion measured using the nominal scale in Set 18 was created. The weight values of the criteria in Set 13-18 and the values related to the alternatives in Set 16 and Set 17 are presented in Table 9. In Table 9, the optimization aspect of criterion K9 in Set 16 and Set 17 is considered as minimum (cost-aspect). Thus, the structure of the problem was changed to include cost-aspect criteria and its effect on the solutions was examined. In Set 18, the color criterion was added as a qualitative criterion and "white, red, blue, white, white, red, white, blue, green, red, white, white, yellow, blue, red" categorical values were assigned from A1 to A15. In this context, solutions can only be obtained with REF in Set 18 because other MCDM methods outlined in the first part of the study need quantitative criteria for the solution of the decision problem. To obtain the solution in K9 in Set 18 using REF, the most frequently observed category was used as the reference category. When the solutions were examined, it was seen that A5 was found to rank first in in sets 13, 14 and 15. The solutions obtained using TOPSIS, VIKOR, and CODAS were found to differ compared to the previous sets. Besides, REF has high rank correlation (rs ≥ 0.8) with other methods in Set 13-Set 17. In Set 18, the nominal (multinomial) criterion was included in the decision problem as criterion K9. Among the methods included in the comparative sensitivity analysis section, only REF can be used in decision problems that include criteria measured on a nominal scale. In the solution obtained with REF, A5 takes first place and A12 takes the last place. The scaling of qualitative criteria is the most adopted approach in the MCDM literature. However, there are also methods that suggest solutions using criteria measured with ordinal scales such as RIM, Regime, and QUALIFLEX. Munda et al. (1994) stated that in the field of MCDM, there is a need for methods that will allow the evaluation of criteria with qualitative (criteria measured with nominal and ordinal scales) or mixed (qualitative and quantitative) data structure together. It can thus be stated that REF, which offers solutions to decision problems with criteria, measured using different types of scales such as nominal (binary, multinomial), ordinal, interval and ratio scale (i.e. mixed data structure criteria) will make a significant contribution to the literature. After analyzing the changing criteria set's effect, the effects of adding new alternatives that have the best or worse performance in the alternative set were examined. There is a fundamental rule that when an alternative has the best performance in all criteria, it should be the first place. For this purpose, Sets 19, 20, and 21 were created to see the effects of adding new alternatives with better or worse values than the existing values in the criteria in Set 1. In Set 19, an alternative with the best values in all criteria was added to the problem. In the same manner, an alternative with the worst values was added to the problem in Set 20. In Set 21, these two artificial alternatives were added simultaneously to the decision matrix.
The performance values of these artificial alternatives are given in Table 10. When the solutions for Set 19-21 were examined, it was seen that all the compared methods ranked the best and worst alternatives correctly. It was determined that the rank correlation was strong (rs ≥ 0.9) with the solutions obtained by other methods in Set 19-21 of REF. The methods that allow the decision maker to set the reference or ideal point as a range are REF and RIM. Set 22, which was created by setting the range 280-360 as a reference in criterion K2 instead of a reference point, cannot be solved with any other methods other than these two methods. A5 was ranked first and A12 last in the solutions obtained with REF and RIM, where the reference is determined as a range. The decision maker may have a different degree of acceptance for values outside the reference point/range. In criterion K2, while the reference point is determined as 360, in the case of the opinion that the values in the range 280-360 are more acceptable than the values lesser than 280, only REF can be used for the solution of the problem. REF allows the decision maker to set different successor points/ranges different from the reference point/range. The unacceptance value (β) must be assigned for each successor point/range determined. In criterion K2, if the reference point is 360, the successor range is determined as 280-360; In order to analyze the effects of the changes in the β on the solution, different β values were used. Set 22-Set 32 were created by increasing the values of β from 0 to 1 in each set by 0.1 for the 280-360 successor range. When the solutions for Set 22-32 were examined, the rankings obtained in Set 1 are seen to be completely preserved if the unacceptance value is determined as 0.6 and higher. The most ranking changes were found in Set 23 and Set 24, which had two changes each. If the value of unacceptance is 0, the reference value becomes a range of 280-360. The solutions show that REF managed to reflect decision maker's preference structure on the solutions through successive points/ranges and unacceptance values. The analysis of parameter changes with REF reveals that the preferences of the decision maker were reflected in the problem. It is believed that reference and successor points/ranges and unacceptance values will be useful in reflecting the decision maker's preferences in solving the problem.

Conclusions
Although MCDM methods have widespread use, it is not possible to ensure effective results in all problems with the same method due to the variety of components in decision problems. The decision makers thus have to choose most appropriate method or methods depending on the nature of the problem. If the criteria are not regarded as measured by an interval scale or having cardinal data type, they are considered as qualitative criteria, the number of methods that can be used to solve the decision problem remains limited. In this point, REF, the new MCDM method proposed in this study, offers solutions on the basis of reference and distance for decision problems having mixed type data structure. REF allowed criteria with different data types to be used together, and also allowed the decision maker to pick the choice closest to the reference points or ranges determined. In the application of REF to the real life problem, the second-hand automobile choice problem was solved. In this problem, criteria measured by different scales such as, nominal, ordinal, interval and ratio scale were used together with REF. In addition, in the color, which is a qualitative criterion, the most frequent categorical value was determined as reference. Also, in some criteria, the ranges that were outside the reference, but that were more acceptable than the other ranges, were reflected to the solution. For this purpose, the usage of different unacceptance values for these successor points/ranges was adopted. Also, a comparative sensitivity analysis was performed some reference-based methods and REF.
In the comparative sensitivity analysis, the effects of changing criteria weights, changing criteria sets, ranking the best and the worst alternatives correctly, changing parameters and using qualitative criteria were examined. In general, it was observed that REF has a high rank correlation with other methods and effectively reflects the preference structure of the decision maker in the decision problem. The lack or absence of any other MCDM methods that allows for the solution of the decision problem, including qualitative criteria and references and successors, attests to the originality of the study. REF is expected to contribute to the MCDM literature and individuals, businesses, non-governmental organizations and public enterprises in their decision problems.