Ranking of building maintenance contractors using multi-criteria decision making methods and an artificial neural network model

Article history: Received: September 11, 2018 Received in revised format: September 11, 2019 Accepted: December 12, 2019 Available online: December 12 2019 Building Maintenance plays an important role throughout the building lifecycle from devising conceptual plans to the end. Due to the high cost of building maintenance and the direct impact of maintenance effectiveness on the quality of life of building occupants, special attention must be devoted. One of the most important issues in this field is building maintenance contractor selection. This issue becomes even more critical in public buildings, such as hospitals, offices, and military centers. The purpose of this study is to present a method that can be used to select the contractor in such a way that the response robustness is high and the employed method is the most accurate one among other similar methods. To do this, the contractors are ranked by 7 multicriteria decision-making methods. Then, the Spearman correlation coefficients are obtained for each pair of methods. When there is a significant difference between the outcomes of the methods, the output of each method is compared with the output of the Artificial Neural Network (ANN) model. The method with the least difference with the neural network output is taken as the superior method. After selecting the best method, a robustness analysis is performed on it to verify the stability of the answer. The proposed model is implemented on a real case study. Statistical analysis shows that the implementation of this method has increased the satisfaction of the residents. © 2020 by the authors; licensee Growing Science, Canada.


Introduction
Building managers need a decision support system to select a maintenance contractor. The task of selecting maintenance contractors is one of the most economically, socially and technically complex decisionmaking processes. The process should evaluate the appropriateness of the infrastructure and performance of the human factors to minimize costs while meeting the standards. The building maintenance costs will be very lower if a building has a reliable maintenance plan which is devised and executed by an expert contractor. Generally, these costs justify the existence of a maintenance plan. An expert maintenance contractor can increase the level of maturity of the maintenance system, decline the maintenance risks significantly, in addition to optimizing the costs. Choosing an incompetent contractor will not only increase costs and reduce the quality of service but also reduce the comfort of residents or clients of the building and may even endanger their lives.
Although it is very important to select a suitable contractor for the public, military, healthcare, and commercial buildings, very few studies have addressed this issue. Therefore, it is essential to conduct a precise study in this field. Unfortunately, most studies conducted in similar fields have not shown that their methods are optimal and produce robust results. In many others, there is no guarantee that the ranking criteria are comprehensive and cover all we need to know about the contractors. This study has attempted to eliminate these deficiencies in previous and propose a reliable methodology.
Another important question raised in this study is what qualities a good building maintenance contractor should have. In other words, what criteria should be considered in the ranking of contractors to make sure that all we need to know about contractors are taken into account, and no unimportant or irrelevant criteria are considered? Lam et al. researched the benchmarking success of building maintenance projects and introduced a set of critical success factors for these projects. The factors are time, cost, quality, functionality, safety, and sustainability. (Lam, et al., 2010) Hossam et al. expressed that having a good maintenance strategy and efficiently implementing that strategy are two characteristics each building maintenance contractor needs to have. They mentioned that a good maintenance strategy sets a preventive maintenance plan, prioritize and schedule maintenance works, does asset lifecycle analysis and replacement optimization, assesses and controls the maintenance execution process, and plans for continuous improvement (Hossam, et al., 2019). Lavy and Bilbo (2009) in a research about the maintenance management of public schools concluded that the maintenance contractors who have comparatively more detailed and contemporary information about the facility's condition, have much better performance comparing to the other contractors. Kivrak et al. (2008) highlighted the role of knowledge management systems in building maintenance and emphasized that contractors must be able to gather individuals with different skills and knowledge to meet all maintenance needs. Njuangang et al. (2015) introduced 53 key performance measures that are needed for conducting a high-quality building maintenance project. Decision-makers can check these elements for each contractor to see which one has been more successful at them based on their performance in previous projects. Cooper (2015), in a study about maintenance management in the social housing sector, concluded that maintenance contractors must be able to conduct their projects economically, environmentally, and socially sustainable. This factor must be considered public buildings, where the environment and people are directly influenced. Khudhair and Isik (2018) wrote a review paper to list all needed criteria for evaluating maintenance contractors. They included a wide variety of financial, quality, societal, safety, functional, and environmental factors in their list.
As mentioned before, the contractor evaluation problem in the building maintenance industry is not exclusively discussed in the literature and no widely-accepted selection framework for it. However, there is a famous model devised that is widely used for contractor selection in the aviation maintenance industry. This model is called the SHELL model which categorize the evaluation criteria into software, hardware, environmental, and liveware groups. This model also considers the interaction of these categories. For example, there are some criteria which influence both liveware and hardware such as Equipment Safety. The second L of SHELL is to highlight the importance of liveware interaction as a critical success factor. This model was first proposed by Elwyn Edwards and later improved into a by Frank Hawkins (1993). Although this model is firstly devised for the aviation maintenance industry, it is a general model and can be employed in other similar industries such as building maintenance. In this research, the SHELL model is used to identify the ranking criteria for the contractor selection problem.
In this study, ranking of the building maintenance contractors has been done by 7 of widely used MCDM methods including ARAS, COPRAS, MOORA, WASPAS, VIKOR, SWARA, and PROMETHEE II. Then, the Spearman correlation test has been done for each pair of methods to find out whether the outcome of them are similar or not. If the results of different methods have low correlations (less than 0.6), we employ an unsupervised ranking method (ANN in recommended) to measure the accuracy of each supervised method. This is a heuristic way of selecting the best ranking method, but there is no scientific proof to show that the selected method produces robust and reliable answers. Therefore, we need to go further and do a sensitivity analysis on the obtained answers. If the sensitivity analysis outcome is not satisfactory for the decision-makers, we can choose the method with the second-lowest distance with the ANN results. This loop will continue until the outcome of the sensitivity analysis be approved by the decision-makers (The result of the selected method is robust enough for making a firm decision about the contractors).
In section 2 of this paper, the SHELL model is introduced in details. Then, the criteria of the ranking process are introduced according to this model. In section 3, the criteria are weighed by the CRITIC method. These weights determine the relative importance of the criteria in the ranking of the contractors. In section 4, the ranking process is carried out for a case study using the 7 mentioned MCDM methods. After that, the statistical tests are performed, and the ANN model is run to choose the best ranking model. In section 5, the robustness analysis is done for the chosen method.

Definition
The SHELL model is a conceptual model of human factors that helps to better understand the relationships between human factors and different subsystems in the aviation industry. (Shanmugam & Paul Robert, 2015) The name of this block model stands on the first letters of its components (software, hardware, environment, environment), and is defined to model human relationships with other components in the air transport system. The SHELL model explores the various factors that affect human relationships with other systems and generate latent and explicit errors. Fig. 1 shows the elements of the SHELL model and the relationship of these elements. (Shanmugam & Paul Robert, 2015) The following relationships usually exist in the SHELL model:  Hardware with Liveware: It is also named man-machine relationship. Maintenance tools and equipment must be easy-to-use, safe, and provide a considerable value of service.  Software with Liveware: These interactions include laws and regulations, processes, training, checklists, behaviors, and interactions of human resources.  Liveware with Environment: This relationship addresses organizational, supervisory, and social aspects of the environment, such as employee morale and organizational health. Noise, heat, and vibration can cause L-E interaction error. These errors can also be minimized by controlling these three factors.
 Liveware with Liveware: This interface is about leadership, employee engagement, and organizational interactions. Experts believe that problems in the sector, such as lack of coordination in teamwork, cause many system failures. Proper human resource management, training of personnel and managers, and knowledge management can help improve this sector.

Application
A contractor who can better control the root causes of maintenance errors is better than others and is chosen as the top option. Therefore, according to the SHELL model and based on expert opinions, we determine the criteria for selecting a building maintenance contractor. To do this, we use the Delphi method. The Delphi cycle continues until experts reach a consensus on the criteria. Figure 2 shows the selected criteria.

Fig. 2. The Ranking Criteria based on the SHELL model
After that, each candidate contractor must be scored based on the criteria introduced in Figure 2. The quality of the scoring process significantly affects the resulted ranking. So, strict instruction and supervision are needed to obtain a reliable outcome. For this reason, standard questionnaires were designed to rate the contractors. The experts were asked to give scores between 1 to 5 to each criterion based on their observations and the previous records of the contractors. The answers should be based on the description written against each score. These explanations help ensure that all experts have the same understanding of the scores they have given. Table 1 shows a sample questionnaire that is devised to rate the internal procedures of a contractor of our case study. Most processes and procedures are standard, taught to responsible persons, and implemented in most projects. 3 Procedures and processes are not available in written form, but most staff have a similar understanding of them.
2 Processes and procedures are not well formulated, there is no common understanding of them, and they are not implemented equally in projects. 1 There is no specific process or procedure, and things are done based on personal decisions. Now, MCDM methods can be executed on the outcome of the surveys. However, before ranking the contractors, the weights of the criteria must be determined. In this study, CRITIC method has been used to do this.

Weighting the criteria
Weights of criteria are influenced by the perception of decision-makers and directly affect the ranking of criteria. Weights are usually determined by decision-makers based on their experience, knowledge, and understanding of the problem. However, this raises doubts about the reliability of the results. Numerical valuation approaches are used to overcome such a problem. The CRITIC method is an objective method for determining the weight of criteria. This method was developed by Diakoulaki et al. (1995). This CRITIC considers the severity of conflict inside a component and incompatibility between each pair of components in a decision problem. These two concepts are defined by correlation coefficients and standard deviations. The standard deviation is related to the values of each criterion, and the correlation coefficient is related to each pair of criteria. (Diakoulaki, et al., 1995). The first step in this method is to form a decision matrix. The decision matrix of this method is similar to the decision matrix of TOPSIS. In this method, positive and negative criteria are not involved in weight determination. Table 2 the decision matrix of this study, which shows the average scores of 8 decision-makers (managerial board of a hospital in Isfahan, Iran) given to 10 candidate building maintenance contractors. The second step in this method is the decision matrix normalization. Eq. (1) is used to normalize this matrix. After the normalization, all elements place within the range of 0 to 1. min max min In the third step, the weights of the criteria are determined. In this process, the standard deviation of each indicator and its correlation with the other criteria are the effective elements. In Eq.
(2), j W is the weight In Eq. (2), j C is the amount of information extracted from the criterion j , which is obtained by Eq. (3).
Based on the explanations given for the CRITIC method, the weights of the criteria have been obtained. Table 3 shows the outcome of the CRITIC method for this case study.

Ranking the criteria
After assigning weights to the criteria, the alternatives can be ranked by multi-criteria decision-making methods. Some of these methods that are employed in this study, as well as their characteristics and their needed inputs, are brought in Table 4.

WASPAS
The WASPAS method was introduced in 2012 and is a combination of the Weighted Sum Model (WSM) and the Weighted Product Method (WPM). In this method the indices are independent, and the qualitative indices must be quantified.

SWARA
In the SWARA method (2010), the relative importance of alternatives are determined for each independent criteria, then the alternatives are ranked, based on relative weights of the criteria.

PROME-THEE
The PROMETHEE method was first introduced in 1986. The developers of this approach aimed to find a fundamental way that improves decision making. Therefore this method is efficient. There is no need to use independent criteria in this method  Decision Matrix  Weights of Criteria  Preference Function Parameters (Brans, et al., 1986) After ranking the alternatives by the mentioned methods, the results of these methods can be compared. Table 5 shows the final ranking of building maintenance contractors. Spearman correlation test can be performed on the obtained results to determine how similar the results of different MCDM methods are. Spearman correlation coefficient was introduced by Charles Spearman English psychologist and statistician in 1904. This correlation coefficient uses their rank instead of using the values of the variables. Table 6 shows the results of this test on ratings provided by different MCDM methods. In this table, the green color means that the results of the two methods corresponding to the matrix elements are similar, and the yellow color of indicates a low correlation between the results of the two methods corresponding to the matrix elements.  Rank  ARAS  VIKOR  MOORA  COPRAS  WASPAS  SWARA  PROMETHEE II  1  CR5  CR5  CR5  CR5  CR5  CR10  CR5  2  CR9  CR10  CR10  CR9  CR9  CR5  CR3  3  CR10  CR9  CR9  CR10  CR3  CR4  CR10  4 CR3 By analyzing the result of the Spearman Test, we inferred that the mentioned MCDM methods give 3 different answers. ARAS, VIKOR, MOORA, and COPRAS methods have suggested similar rankings. The result of SWARA and PROMETHEE II methods are similar too, and the ranking proposed by WASPAS is different from the other methods. The next step is to find out which ranking is more reliable and can be considered as the final ranking of the contractors. In this study, an artificial neural network is used to select the best method. The proposed ranking of each MCDM method and the decision matrix are given to the ANN model. The model gives the weight of each criterion. So we can compare the input weights of each method with the weights given by the ANN model. The closer the weights of the MCDM method to the ANN model are, the more reliable that MCDM method is.

Finding the best ranking
Artificial neural networks are built on the insights of neural networks and simple interconnected computing units called neurons that work in parallel. These networks are one of the methods that are capable of doing nonlinear estimation and provide a flexible computational framework for solving a wide range of nonlinear problems. One of the advantages of ANN models over other nonlinear models is that these networks are global estimators that can estimate almost any kind of function accurately. ANN models do not require any assumptions about the model's shape in the modeling process and are generally a data-driven model (Zampighi, et al., 2004). Usually, a single neuron is not enough to solve engineering problems with a large number of inputs. For this reason, neural networks that are composed of multiple neurons or multilayers working in parallel are used to solve such problems. Multilayer networks are very powerful estimators, for example, a two-layer network with a sigmoid layer and a linear layer. The second line can estimate any function. In a multilayer neural network, each layer has its weight matrix, bias vectors and outputs, and the output of each middle layer as the input of the next layer. (Gardner & Dorling, 1998). As mentioned earlier, a perceptron multilayer neural network model is used in this study to estimate the weight of the criteria while assuming the rank of the contractors (Based on the results of MCDM methods). The first layer contains the covariates. In this problem, the covariates are the criteria. The second layer contains virtual neurons that are added to the model automatically for better estimation by SPSS software. In each layer, a bias variable is also added to increase the precision of the estimation.
The main variable is also the rankings given by each of the MCDM methods. Fig. 3 shows one of the ANN models used in this study in which the main variable is the result of ARAS.

Fig. 3. Multilayer perceptron neural network
The outcome of this model is the weights of covariates (criteria). The neural network model is run for all MCDM methods. The obtained weights must be compared to the input weights. The block distance (Manhattan) was used to measure the difference of the estimated weights with the actual weights. Table  6 shows the weights obtained by the ANN model for each of the MCDM methods. The last column of this table shows the block distance of the weights of each method with the actual weights. The results show that the weights of WASPAS method have the least distance with the actual weights. Therefore, we take the proposed ranking of this method as the final ranking of contractors.

Robustness measure
The sensitivity analysis of multi-criteria models has been carried out by several researchers and from various perspectives. Wolters and Mareschal (1995) did this analysis with three approaches: 1. Sensitivity analysis for the rankings against the change in the evaluation of all alternatives for a particular index 2. Sensitivity analysis for the rankings against changes in the evaluation of a particular alternative 3. The minimum change in the weights of the criteria that will lead to the change of the best alternative In this study, the concept of Robustness is used, which means measuring the stability of answer in MCDM models when the weight of the decision-making criteria changes. An appropriate answer does not change with low variations in inputs, and its sensitivity to input parameters is not high. As errors are possible in the input data, if the sensitivity of the model is high, the obtained answers are not reliable. Suppose the initial weight of criterion i ( i  ) is increased or decreased by units. So, the new weight of this criterion will be equal to ( i i       ). The weights of the other criteria are obtained using Eq. (4).
The Robustness Measure ( RM ) is equal to the average of the Spearman and Kendall Correlation Coefficients. This measure ranges from 0 to 1 (See Eq. (5)). Here, each criterion is increased by 15%, 30%, 45%, 60%, 75%, and 90%. The Robustness Measure for the WASPAS method is 0.859 which is acceptable amount of robustness according to the literature.

Conclusion
The SHELL model is one of the most well-known models for identifying risks associated with human factors in aircraft maintenance. In this study, this model was first used to select a building maintenance contractor. Ten general criteria for contractor selection were selected based on the Shell model, and ten contractors who were nominated to manage hospital maintenance were compared with these criteria. The ranking process was performed by seven of widely used MCDM methods including ARAS, VIKOR, MOORA, COPRAS, WASPAS, SWARA, and PROMETHEE II. The Spearman correlation test result showed that the result of these seven methods are not similar, and the decision-makers needed to find out which answer must be taken as the final ranking of the contractors. Therefore, a heuristic method was proposed in this study to overcome this problem. As the weights of the criteria had previously been determined by CRITIC method, and these weights had been used in all seven MCDM methods as an input, a multilayer perceptron model was employed to reverse the ranking process and find out which method had proposed a better ranking. The MLP model takes the rankings and the decision matrix and estimates the input weights. The closer the estimated weights are to the actual weights, the better is the proposed ranking of that method. The WASPAS method had given the best answer according to the proposed method. Also, the robustness of the given answer was checked to make sure that this answer is not too sensitive to the input weights. According to the WASPAS, the contractor number 5 was appointed to manage the maintenance of the mentioned hospital. After 5 months, the customer and personnel satisfaction indices were measured separately. The customer satisfaction indexed was enhanced by 22.1%, and the personnel satisfaction index improved by 39.2%.