MCDM based evaluation and ranking of commercial off-the-shelf using fuzzy based matrix method

In today’s scenario, software has become an essential component in all kinds of systems. The size and the complexity of the software increases with a corresponding increase in its functionality, hence leads to the development of the modular software systems. Software developers emphasize on the concept of component based software engineering (CBSE) for the development of modular software systems. The CBSE concept consists of dividing the software into a number of modules; selecting Commercial Off-the-Shelf (COTS) for each module; and finally integrating the modules to develop the final software system. The selection of COTS for any module plays a vital role in software development. To address the problem of selection of COTS, a framework for ranking and selection of various COTS components for any software system based on expert opinion elicitation and fuzzy-based matrix methodology is proposed in this research paper. The selection problem is modeled as a multi-criteria decision making (MCDM) problem. The evaluation criteria are identified through extensive literature study and the COTS components are ranked based on these identified and selected evaluation criteria using the proposed methods according to the value of a permanent function of their criteria matrices. The methodology is explained through an example and is validated by comparing with an existing method.


Introduction
The concept of CBSE which has a great influence on the software development primarily concerned with the integration of a number of COTS components in the desired system.COTS are usually the reusable software products that are developed for specific use by the various vendors and can be used without any modification for the development of a new system.CBSE approach can reduce the software complexity by decomposing the large software project into small functional or logical pieces that are more manageable with the aim of reassembling them to get the desired system.Here, instead of building the whole software from a scratch, a new system can be assembled and constructed by the use of COTS components provided by different vendors in the market.The COTS based software development has five main phases as shown in Fig. 1 (Oberndorf, 1997).

Fig. 1. COTS-based software development process
The need for rapid software development and requirement of cost, time and effort reduction has increased the usage of COTS components (Alves & Finkelstein, 2003).COTS are defined as the software pieces that can be further reused by the software developers to build the new software systems (Firesmith, 2005;Vigder et al., 1996).Gupta et al. (2012) described the COTS products that are available for purchase, lease or license to the general public.The COTS products are ready made and can be used by the software developers "as it is" and can be easily installed and incorporated with the existing system components.The use of COTS products provides many potential benefits as: (i) Reduction in development cost (ii) Reduction in development time and effort (iii) Improved quality of target software.In spite of many potential benefits, some disadvantages are also associated with the COTS products as (i) Incomplete knowledge of inner working code (ii) Compatibility issues (iii) Unavailability of correct and complete specifications.The COTS component selection involves a number of functional and non-functional criteria.Further, the importance of each evaluation criteria is different and also may vary under different requirements.The nature of the evaluation criteria is always conflicting and imprecise.Hence, there arises a need to develop a method which is efficient for optimal selection and ranking of the COTS components considering the conflicting and imprecise nature of the mutually interacting evaluation criteria.To resolve this problem, the present research paper proposes a fuzzy based matrix method for ranking and selection of COTS components.It is a hybrid method comprising fuzzy logics and matrix methods.Fuzzy logics are used to accommodate the conflicting and imprecise nature of evaluation criteria and vagueness of mind.Key elements in fuzzy logics are linguistic variables or fuzzy numbers rather that integers, (Zadeh, 1965).The matrix method is capable to model multi-criteria decision problems in the form of matrices and then matrices are converted into a single numerical value through its determinant or permanent.The main purpose of performing this empirical research is to improve the understanding of the COTS selection and evaluation process that may have a great influence on the software development.Though the matrix method has already been used for solution of MCDM problems of another area, however, this hybrid approach is being applied for the first time for ranking and selection of COTS.The method is validated by comparing with other existing methods.
The rest of the paper is structured as follows: in section 2, a review of COTS selection and the evaluation problem with the various COTS evaluation criteria and existing COTS selection approaches is provided.Section 3 explains the proposed fuzzy matrix method (FMM).Ranking procedure and empirical study are given in section 4. Results are provided in section 5 whereas a comparison of FMM proposed in this research with other existing methods is given in section 6.The research work is concluded in section 7.

Previous Related Work
COTS selection for a particular application from the large number of available COTS components in the market has become a challenging task for the software developers because the improper selection of COTS may lead either development of poor quality software or its complete failure to perform desired tasks.Various considerations such as software design, software functionality and economics need to be considered before a suitable COTS component is selected for the development of the desired software.So, the research on COTS selection has very important theoretical and practical meaning, whether COTS selection is of rational concern for the better software development.Therefore, the available literature is revealed for selection criteria identified and used by various researchers in the past, methods and techniques developed for COTS evaluation and selection and other general issues relevant to the present research problem.Bertoa and Vallecillo (2002) proposed a quality model for the component based software development (CBSD) based on ISO 9126, that defines a set of quality attributes as Functionality, Reliability, Usability, Efficiency, Maintainability and Portability, etc. and their associated metrics for the effective selection and evaluation of COTS components.Wanyama and Homayoun (2005) found that the COTS selection is a complex multi-criteria decision making (MCDM) problem characterized by uncertainty, complexity, multiple stakeholders and multiple objectives.In the contemporary work Yang et al. (2005) present vendor dependence as a major factor in COTS selection.

COTS evaluation criteria
Huan-Jyh Shyur (2006) identified cost, supplier's support, technological risk, closeness of fit to a company's business, ease of implementation, flexibility to easy change of business and system integration (CO, SS, TR, FB, EI, FC, SI) as the potential criteria for COTS selection.Neubauer and Stummer (2007) solved the problem of COTS selection by categorizing the different COTS selection factors, mainly in 4-groups, namely (i) functional criteria; (ii) quality criteria (defect rate, performance, usability, security, etc.); (iii) strategic criteria (cost, available time etc.) and (iv) domain and architectural criteria.

Table 1
Description of 4-major groups of COTS evaluation criteria Quality Characteristics Quality characteristic is a set of properties of a software product or cots component by which its quality can be described and evaluated.In the past, many researchers have introduced various quality models such as ISO 9126, ISO 14598 etc. for the assessment of the software product quality.The attributes which are considered by various researchers in their quality model are functionality, reliability, usability, efficiency, maintainability, and portability.

Technology Factors
Technology factors are concerned with the technology opted during the development of any component provided by any vendor.

Domain and Architectural Factors
Domain and Architectural factors describe the scope, applicability and the basic structure of the components.

Strategic Factors
Strategic factors are those non-functional requirements that are directly related to the component providers and the organizational policies such as vendor capabilities, business issues and the cost.Wanyama and Far (2008) addressed the problem of COTS selection using reliability, maintainability, security, portability, compatibility, vendor ability, initial product price, initial hardware price, implementation costs, training costs, license conditions as selection criteria.Suleiman (2008) considered system integration interface, functionality aspects, COTS vendor maturity, conformity to system environment (consistency between system requirements, hardware, software application systems and COTS component infrastructure), budget, time, vendor support as selection criteria.Ibrahim et al. (2009) proposed the selection criteria for the COTS component such as usability, security, functionality, performance, recoverability and impact.(2007), Jung and Choi (1999), Huan-Jyh Shyur (2006), Wanyama and H. Far (2008), Basem Suleiman (2008), Rao and Rajesh (2009), Kaur andMann (2010), Indumati et al. (2011), Gupta et al. (2013).

COTS Selection methods
In the component based software development, the selection of appropriate COTS products is of great importance.Many precision-based methods for COTS selection have been developed.J. Kontio (1995) proposed the OTSO approach for the COTS selection and evaluation with the help of AHP (Analytical Hierarchical Process) and WSM (Weighted Score Method).Morisio and Tsukios (1997) introduced a two-phase model "Isuware" based on multi-criteria decision making approach to the evaluation of COTS components Lichota et al. (1997) proposed PRISM (Portable, Reusable, Integrated software modules) based on progressive filtering comprising of two parts as formation of generic component architecture and product evaluation process (PEP) for the COTS selection.Tran and Liu (1997) proposed a waterfall stylistic approach CISD for the identification and evaluation of COTS components and then their integration into the system for the software development.Maiden and Ncube (1998) proposes a new process PORE (Procurement-Oriented Requirements Engineering) to address the lack of guidance for acquiring requirements to enable evaluation of commercial off-the-shelf (COTS) software.Kunda and Brooks (1999)  (2013) adopted Neuro-fuzzy approach and presented an effective methodology ANFIS for the COTS component selection.Sarkar (2012) provided a comprehensive framework ACT (Architecture Centric Tradeoff) for the COTS selection, implementation and governance by focusing on benefits to the customer organization and system Integrator.Gupta et al. (2013Gupta et al. ( , 2011) ) has developed a systematic and interactive Fuzzy Optimization framework for the solution of bi-objective fuzzy COTS selection.Kaur and Singh (2014) used PROMETHEE whereas Nazir et al. (2014) used Analytic Network Process approach for the selection of COTS.A brief description about the various COTS selection methods is summarized in Table 3.

Methodology Adopted
In this paper, a multi criteria decision making approach combining fuzzy set theory and matrix method is proposed for optimal evaluation and ranking of COTS components based on a number of conflicting evaluation criteria.Optimal refers to the best under given circumstances of certain parameters or restrictions.Here, the optimal selection refers to an appropriate selection of COTS candidates for a particular application from a group of viable and potential COTS candidates based on pre-identified evaluation criteria considered in this study.The concept of optimal selection is explained with the help of following example in its simplest form.
Example: A class consists of 60 students.The top ten students, in accordance to their academic merit, are S1, S2, S3… S9, S10.Five students are to be selected to represent the institute for a competitive event considering the sole criteria of academic merit.In general, top five students, i.e.S1, S2, S3, S4 and S5 will be selected.If, say, two students S2 and S4 are not available due to ill health or any other family engagements/problems for participation, then the selection will comprise S1, S3, S5, S6 and S7 students.
It is called as optimal selection.However, in this case, only one criteria, i.e. academic merit is considered and if more numbers of criteria e.g., academic merit, medical fitness, physical health, decision making, team management, etc. are to be considered for selection; it will become more complex and hence need to be modeled as an MCDM problem.
The matrix method has already been used for the ranking of software engineering metrics (Garg et al., 2013) and power plants (Dahiya et al., 2007).This hybrid methodology takes the advantage of both fuzzy set theory to accommodate impreciseness and vagueness and matrix method for calculation of single numerical index called permanent for ranking of COTS components.A brief introduction to basic concepts of fuzzy logics, algebraic operations, triangular fuzzy numbers, linguistic variables, and matrix operations is presented in this section.

Fuzzy Set Theory
Fuzzy set theory is referred as an extension of classical set theory where the elements of set have varying degree of membership.Any logic that is based on two values either true or false seems to be inadequate while describing human reasoning.In contrast, fuzzy logic can use the whole interval between true (1) and false (0).Simply it can be stated that fuzzy logic is derived from FST to deal approximations rather than precision.Fuzzy logic is a form of many valued logics where the truth values of variables fall between 0 and 1 and are considered to be "fuzzy".It does not happen in case of Boolean algebra/logic in which the truth values are either 1 or 0, often called "crisp" values/score.The values 0 and 1 are the extreme values in case of fuzzy logic and it also takes the in-between values, for example, the result of a comparison between two things could be not "tall" or "short" but " 0.38 of tallness."It means more number of truth values, depending on fuzzy scale, will lie between extreme values.In case where linguistic variables are used, these in-between values are expressed by specific (membership) functions.Zadeh (1965) introduced initially the fuzzy set theory by considering the problem of fuzziness of the data.Fuzzy set theory basically provides a better way to solve the various problems having some uncertainty in them.Mathematical operations can easily be applied to fuzzy sets (Dubois & Prade, 1979).

Triangular fuzzy numbers
Triangular fuzzy numbers (TFN) are used as membership functions, corresponding to the elements in a set, as shown in Fig. 2. TFN is most accepted among the various shapes of fuzzy numbers and also most widely used because of its capability to represent uncertain information.The use of the triangular fuzzy numbers is intuitively easy for the decision makers to use and calculate.A fuzzy number is a triangular fuzzy number if its membership function can be denoted as follows: where a, b and c are real numbers and c = a = b.To calculate membership functions for different operations, Zadeh's extension principle can be used.Only addition and multiplication are used here.Defining triangular fuzzy numbers A1 and A2 by the triplets as A1 = (c1, a1, b1) and A2 = (c2, a2, b2), addition and multiplication operations of A1 and A2 can be expressed as follows: Addition: if  denotes addition.

Linguistic terms of triangular fuzzy numbers
Fuzzy set theory is primarily concerned with quantifying the uncertainty in human thoughts and perceptions, where linguistic terms can be properly represented by the approximate reasoning of fuzzy set theory.The weights of various criteria and the rating values of COTS components are considered as linguistic terms throughout this paper.A linguistic term can be defined as a variable whose values are words or sentences in natural language.In this research paper, a seven point fuzzy scale is used to evaluate the weights using linguistic terms such as extremely more important (EMI), very more important (VMI), more important (MI), important (I), less important (LI), very less important (VLI) and extremely less important (ELI).To determine the suitability of different COTS versus different evaluation criteria, the rating values in a seven point fuzzy scale can be accessed by linguistic terms such as very high (VH), high (H), above average (AA), average (A), below average (BA), low (L) and very low (VL).The seven point fuzzy scale is shown in Fig. 3.

Fig. 3. Seven point fuzzy scale
The linguistic variable along-with their corresponding membership functions and crisp scores for a seven point fuzzy scale are given in Table 4.

Fuzzy algorithm for COTS ranking problem
A systematic approach to the COTS ranking problem, based on fuzzy set theory and multi-criteria decision analysis, is described in this section.Various methods have been proposed to aggregate the expert's opinions such as mean, median, max, min and mixed operators.In this study, the mean operator was used to aggregate the assessments of experts.For COTS ranking problem, there is a group of 'n' experts (E1, E2, . . .,En), who evaluate the weights of k criteria (C1, C2, . . .,Ck) and the ratings of m COTS components (A1, A2, . . .,Am), under each of these k criteria.Let Wte (t = 1, 2,. . ., k; e = 1, 2,. . ., n) be the weight given to Ct by expert Ee.Let Rite (i = 1, 2,. . ., m; t = 1, 2,. . .,k; e = 1, 2,. . ., n) be the rating assigned to COTS Ai by expert Ee under criterion Ct.Wt and Rit are defined as follows: where Wt is the average weight of criterion Ct and Rit is the aggregated rating of COTS components Ai under criterion Ct.

Conversion of fuzzy numbers to crisp scores
As the aggregated assessments are represented as triangular fuzzy numbers, there is a requirement of a conversion mechanism to convert these fuzzy triangular numbers to a single numeric value known as crisp value/scores.In this paper, the maximizing set and minimizing set methods (Chen, 1985) are applied.If ( 1, 2,..., )   F i m i  be the fuzzy ratings of m COTS components.Chen (1985) defined the maximizing set and, minimizing set Further, the right utility value ( ) U F i M and the left utility value for COTS metric 'i' are defined as: And, the total utility value ( ) T i U F of alternative 'i' are defined as:

Matrix Method
COTS components are characterized by multiple criteria, which need to be converted into a single number index.This single number index will be used to rank the COTS.This value for each COTS component is obtained using the concept of matrices.The crisp scores of the aggregated assessments i.e. ratings of the COTS components and the relative aggregated weights of all evaluation criteria are stored in a matrix known as 'Criteria Matrix' having size n x n corresponding to n criteria.The diagonal elements (aii's or ai's) and the off-diagonal elements (aij's) of this matrix give the aggregated ratings of different COTS versus different ranking criteria and the relative aggregated weights of different evaluation criteria, respectively.Thus, the criteria matrix is a combination of two matrices namely 'COTS Rating Matrix and 'Criteria Weight Matrix'.
Criteria Weight Matrix: The Criteria Weight Matrix is formed by considering aggregated weights of different criteria.The off diagonal elements of this matrix show the aggregated weights of the criteria e.g. the element (aij) of this matrix will give the relative importance weight of j th criteria in respect of i th criteria.All diagonal elements of this matrix are zero because there is no significance of comparing a criterion with respect to itself.Mathematically aij = weight of j th criteria/ weight of i th criteria- COTS Rating Matrix: This matrix is formed on the basis of deterministic values (crisp values) of the aggregated ratings of the COTS matrices versus different ranking criteria.This is a diagonal matrix whose elements (aii's or ai's) represent the aggregated ratings of different COTS components versus different evaluation criteria.
Criteria Matrix: The criteria matrix is formed by combining the criteria rating matrix and weight matrix.The criteria matrix is a complete representation of the COTS selection system which considers the performance ratings of the COTS for each evaluation criteria as well as the relative importance of the various evaluation criteria under considerations in an integrated manner.Thus the 'Criteria Matrix' corresponding to 'n' criteria, in general, is written as:

Permanent Function Representation
In order to compare and rank the various COTS candidates, it is necessary to have a single numerical index for each criteria matrix and the permanent function representation is the method to convert the criteria matrix into the single numerical index.Since, no negative value appears in the permanent function representation and hence there is no chance of loss of any information contained in criteria matrix.Variable Permanent Function (VPF) is a standard matrix function that is used in combinatorial mathematics.It is a powerful tool for multi-criteria based evaluation and ranking of the systems in some specific order.VPF is similar to the determinant of a matrix with a difference that no negative term appears in it.Further, computer software is developed to determine the value of the Permanent of the 'Criteria Matrix'.The algorithm is: .

COTS Components Searching and Screening
Initially, a search process is performed to find the suitable COTS components based on the availability of the documentation that describes it, the familiarity of the community and its domain of applicability.
An assumption is made that the COTS components actually considered fulfill all functional requirements of the target system.

Formation of Evaluation team
In the existing COTS selection and evaluation literature, the data that can be used for ranking of preselected COTS component are unattainable, due to lack of maturity in this field.So, the reliance on the expert's opinion was the optimal approach to the problem of collecting selection and evaluation data for COTS components.The first step is to form a team of experts who represent a wide variety of experiences as is obtained in universities/consulting firms, software companies, or government agencies demonstrated by publications, hands on experience and managing research in the area of component based software engineering related to the issues under study and should also be versatile enough to address these issues.Some of the experts have been both in academia and in company.
Members of the software companies may have better insight into the issues of this study, whereas academia may have better measures of knowledge in experimental development and it is at the edge of technological advances.For this research, finally a team of ten experts was formed deliberately.All experts have more than 20 years experience in the area of component based software engineering.Out of these ten experts, six are from software industries, three from academics, and one of the software research laboratories.

Selection of Evaluation criteria
COTS components can be compared with the mean of several attributes, named as evaluation criteria.
In this study, the evaluation criteria are categorized into four major categories as Quality, Technology, Strategic and Domain & Architectural factors.Each of the evaluation criteria relates to some particular aspects of the measure considered important to the objectives of the study.In this study, we choose the strategic factors criteria for the selection and ranking of COTS components as shown in Fig. 4.

Fig. 4. COTS selection criterion
Strategic factors alone are considered in the present research work because all other criteria/factors are functionally required and hence must be fulfilled.Moreover, the strategic factors are the most common overlooked issue in the available literature.The available budget is very important which forces to consider the 'cost' criteria.Further, these criteria 'vendor capabilities' and 'business issues' play a vital role, even after the selection has been made.The brief description of the evaluation criteria considered in this study is provided in the Table 5.

Financial Condition
Financial condition deals with the financial position of the vendor in the market, i.e. "How much the vendor is financially strong"?Implementation & Serviceability Implementation and Serviceability are the property which represents the support of the implementation and further use of COTS provided by the vendor.

Licensing Arrangements
An End User License Agreement (EULA) is a legal contract between a software application author or publisher and the user of that application.

Organizational Policies
An organizational policy is a formal document describing the organization's position on a particular aspect of compliance with regulations, standards, and guidelines.

Risk factors
How much risk is involved if the particular COTS component is used?Cost

Production Cost
The cost for the system at the time system fulfilling its expectations.

Installation & Implementation Cost
The cost incurred to make the system available for use.

License Cost
The cost for the permit from vendor to install the system.

Upgradation & Maintenance Cost
The cost incurred during the useful period of the system.

COTS evaluation of the fuzzy based matrix method
In the proposed methodology, the importance of the evaluation criteria and the performance ratings of the COTS components are assigned into linguistic variables by the experts.The linguistic variables are then converted into corresponding fuzzy numbers, triangular in nature, and then into crisp scores.The 'Criteria matrix' for each component is obtained using the aggregated average weights of the evaluation criteria and the performance ratings of the respective COTS component.The permanent value of each matrix is then determined in order to rank various alternate COTS components in ascending/descending order of merit.The COTS component having the highest value of the VPF is ranked at number 1, the next as number 2, and so on.

An Empirical Study
Databases are used to maintain internal records that support many business activities.With the rapid changes in computer technology and the decline of hardware costs, many applications are becoming database management systems.The database management system is specialized software which embodies the technology and the knowledge developed in the past several decades from where the supremacy of the databases is derived.A DBMS is a dominating tool which is capable to handle even very huge data, to control it efficiently and to store it for long periods of times.In this case study, the developed method is applied to select the best DBMS candidate to develop a website of an online store.The complete procedure is illustrated in the following steps:

COTS Candidates: Selection and Performance Rating
To select the COTS candidates related to database management system for the development of a website of an online store, initially the most popular and widely used websites were visited, namely: www.flashline.com;www.jars.comand www.componentsource.com.A number of possible DBMS COTS candidates were identified.The identified COTS candidates are then screened on the basis of the version, availability and compatibility etc.The candidates with obsolete version and no longer available in the open market are omitted.The potential COTS candidates are then shortlisted after brainstorming with the evaluation team of ten evaluators/experts.These evaluators are from the fields of information technology, database experts and academic researchers who are specialized in database system in various reputed industries and academic institutions and are termed as "Experts".The COTS candidates finally selected in this research are: (1) Oracle; (2) SQL Express; (3) MYSQL; (4) MS-Access and (5) Postgreys.

Data Collection
The questionnaire is considered to be the best way of collection of data in survey researches which are exploratory in nature.Since, no secondary data are available in the open literature and hence questionnaires are used to collect primary data.Two questionnaires were prepared.The first questionnaire is related to estimating the weights of the COTS evaluation criteria and the second questionnaire is used to get the performance ratings of the COTS components based on each evaluation criteria.The weights and ratings are obtained on a seven point fuzzy scale.The questionnaires are designed in three parts:  The first part of each questionnaire contains a covering letter which explains the purpose of the research study and statement of confidentiality.
 Second part consists of demographic details e.g.name of the organization, field of expertise (software development, software design, software purchasing and procurement, etc.); length of experience, qualification and designation, etc. of the respondent expert/evaluator which are necessary to establish his credibility and suitability for the research work to be undertaken.
 Third part of the questionnaire consists of assigning weights to the evaluation criteria/performance ratings of the COTS components on a predetermined seven point fuzzy scale.
Two separate questionnaires were designed.The first questionnaire is used to get the weights of the identified evaluation criteria from the experts.Here, the weight of evaluation criteria means the level of importance of that criterion in the selection of any COTS component for the development of particular software.Since, all evaluation criteria are not equally important, it is necessary to find out their own importance.Ten experts were asked to rate the level of importance of each selection criterion considered in this study on a 7-point fuzzy scale in linguistic terms, namely: Extremely More Important (EMI), Very More Important (VMI), More Important (MI), Important (I), Less Important (LI), Very Less Important (VLI) and Extremely Less Important (ELI).The following principles are followed in assigning the weights to the criteria/sub-criteria:  'Extremely More Important' a linguistic term is assigned if the criterion is of greatest importance, whereas the linguistic term 'Extremely Less Important' is assigned if the criterion is of least importance.The in-between linguistic term is assigned based on the comparative importance of the criteria.
All questions are completely independent and sample questions are given in the Appendix.The weights of the criteria assigned by the experts in linguistic terms on a seven point fuzzy scale are given in Table 6.

Table 6
Criteria weights assigned by the Experts in linguistic terms on a 7-point fuzzy scale These linguistic terms are converted into TFNs in accordance to Table 4.The TFNs of all the ten experts are then aggregated and averaged arithmetically i.e. simple arithmetic mean is calculated using Eqs.(4-5).The TFN corresponding to the arithmetic mean is then converted into a single numeric value known as crisp value using Eqs.
(5-10).For example: the crisp value for the importance of the criterion 'Market Trends' is calculated as given in Table 7.The arithmetic mean TFN is then converted into crisp value using Eqs.(6-10).The corresponding crisp values for the importance of all 14 criteria are obtained in the same manner.The weight of each criterion Then, performance ratings assigned by the experts are converted into triangular fuzzy numbers and crisp values through aggregation and arithmetic averaging as explained in the above paragraph.The values, so obtained, are given in Table 10.
Finally, the ranking values of all five COTS components and their respective rankings are obtained and presented in Table 11.In order to validate the methodology and the results obtained, the present COTS component selection problem using the same datasets is solved using Distance based Approach (DBA) proposed by Kumar and Garg (2010).The results so obtained are given in Table 12.The comparison shows that the results obtained using the proposed method, i.e. the fuzzy matrix and DBA are same for the evaluation criteria -cost and strategic concerns.In case of evaluation criteria, namely 'Vendor Capabilities' and 'Business Issues' a minor difference in the rankings of two COTS components is observed.In case of fuzzy matrix method MYSQL is ranked at #1 followed by POSTGREYS whereas the Postgreys are ranked at #1 followed by MYSQL using DBA method.The reason is quite obvious that there is a negligible difference in the ranking values (4097.519and 4095.037) of these two components using the fuzzy matrix method.The same pattern is observed in the ranking values, SQL Express and MS-Access components.It is thus evident that the fuzzy matrix method is validated.

Results
According to the methodology adopted in this empirical study, the higher the value of permanent implies better ranking.The comparative rankings of all five COTS components based on vendor capabilities, Business issues and cost individually as well as taken altogether is given in Fig. 5.It is depicted that MYSQL has been ranked first on the basis of vendor capabilities, Business issues and cost as evaluation criteria and is followed by Postgreys and SQL Express ranked at 2 and 3 and are top three DBMS cots components.Oracle is ranked at five based on Business issues and cost whereas ranked at four based on vendor capabilities which shows the least use of this component.MYSQL is ranked at #1 based on strategic concern, i.e. considering all three evaluation criteria, namely vendor capabilities, Business issues and cost collectively, and is followed by Postgreys whereas Oracle is ranked at the last.Hence, Postgreys can be termed as an optimal COTS selection for this kind of problem considering all constraints inspite the fact that MS Access is a better option in terms of 'cost' than Postgreys.This paper addresses the issue of ranking the COTS using the state-of-art knowledge in the field of component based software engineering by applying fuzzy based matrix methodology.A set of evaluation criteria is identified; their weights and ratings are taken from various experts in linguistic terms.The following conclusions can be drawn from the study:  As the triangular fuzzy numbers concept of fuzzy set theory is used in this study, the data collection, calculation and result interpretation become very easier for the experts due to the fact that FST is capable to accommodate the vagueness of mind and imprecise nature of the evaluation criteria. Interdependencies of the evaluation criteria have given a due consideration in the proposed methodology.The non-diagonal elements of the criteria matrix represent the interdependencies of the evaluation criteria which influence the value of the permanent function which is used to rank the alternate COTS components. A computer based decision support system (DSS) is developed for ranking of COTS on the basis of various evaluation criteria which is very user-friendly and do not require much technical knowledge.Once a complete database is prepared in terms of evaluation criteria/sub-criteria to be considered and the COTS components, the DSS may be used to rank the COTS components.
In terms of future work, it is necessary to rank the COTS components based on all four groups of evaluation criteria which includes both functional and non-functional requirements.Further, in case of large software systems, COTS components which are mutually dependent, should not be evaluated individually as single and local evaluation and selection may lead to sub-optimal solution from a global perspective.

Fig. 2 .
Fig. 2. Membership function of a triangular fuzzy number b a c

Fig. 5 .
Fig. 5. Comparative rankings of COTS components What is the level of importance of 'Market Trends' for selection of DBMS COTS for the development of a website of an online store?o Extremely More Important o Very More Important o More Important o I m p o r t a n t o Less Important o Very Less Important o Extremely Less Important Questionnaire -II Q-2 To what extent the organizational policies are followed by the vendor of the COTS component "Oracle".o Very High o High o Above Average o A v e r a g e o Below Average o Low o Very Low Q-3 What the level of 'R & D Technology' being adopted by the vendor of "MYSQL?o Very High o High o Above Average o A v e r a g e o Below Average o Low o Very Low Q-4 How you rate the COTS component "Postgreys" for its 'Upgradation and Maintenance Cost'?o Very High o High o Above Average o A v e r a g e o Below Average o Low o Very Low

Table 3
Existing COTS selection approaches

Table 4
Linguistic terms representation using triangular fuzzy numbers

Table 5
Definitions of Criteria/Sub-criteria

Table 7
Crisp value calculation

Table 11
Ranking values and ranks of COTS components using the Fuzzy Matrix method

Table 12
Overall Ranking of DBMS COTS Component using DBA