Framework for Evaluating Quality Performances of Subcontractors: Case of Turkish Contractors *Befrin

Success of a construction project mainly depends on the performances of the subcontractors. Thus, general contractors should be very careful when selecting their subcontractors. Turkish contractors are active in international markets, where competition is fierce. This study aims to provide general contractors, who predominantly operate in international markets, with a practical and user-friendly subcontractor quality performance measurement framework. The methodology of this study has two main phases, pre-survey and survey stages. The pre-survey stage aims to identify the most important key performance indicators (KPIs), which can be used to develop a framework for measuring the quality performances of candidate subcontractors. For this purpose, an extensive literature review and a questionnaire survey among 40 large scale Turkish contractors were conducted. In the survey stage, the performance measurement framework was developed using the group decision on the weights of the most important KPIs obtained from the analytic hierarchy process (AHP) calculations. Face-to-face interviews were conducted with the target users of the proposed performance measurement framework to construct the comparison matrices and thereby determine the weights of these KPIs. The proposed framework can be used by Turkish contractors to measure the quality performances of subcontractor candidates in an objective, systematic and structured manner.

components and assign subcontractors to them based on their speciality areas. In construction projects where large portions of the activities are outsourced, numerous subcontractors are obliged to work together, which in turn makes construction sites chaotic and complicated. In such projects, coordinating and controlling the works of these subcontractors is not an easy task for general contractors. Since most of the construction activities are carried out by subcontractors and the success of the entire project, which is mostly assessed in terms of time, cost and quality, is determined by their performances, working with ineligible subcontractors may bring about failures such as delays, cost overruns, quality problems, disputes, etc. Therefore, general contractors should be very careful during the subcontractor selection process and select the right subcontractor for the right job to achieve business continuity in such a highly competitive market.
In practice, general contractors tend to select the subcontractor who offers the lowest bid price (Tserng and Lin, 2002;Arslan et al., 2008;Mbachu, 2008;Hartmann, Ling and Tan, 2009) or is known from previous projects (Tserng and Lin, 2002;Arslan et al., 2008;Ulubeyli, Manisali and Kazaz, 2010;Choudhry et al., 2012). However, both practices may lead to severe problems in either meeting quality requirements or controlling the cost. It is obvious that there is a need for a sound and systematic approach for measuring/evaluating the performances of candidate subcontractors and selecting the most appropriate one in order to fulfil the contract requirements and ensure business continuity.
General contractors may take into account several tangible and intangible criteria during the subcontractor evaluation and selection process. While some of these criteria such as time and cost can be assessed easily as they can be numerically expressed, the remaining criteria such as safety and quality performances of the candidate subcontractors cannot be easily assessed as they are highly subjective. The quality of construction works is defined as "to be fit for use as intended" (Trinkūnienė et al., 2017) and it is very difficult to estimate and evaluate the quality performance of a subcontractor at the beginning of a construction project. Moreover, even though subcontractors perform the large portions of the construction works, general contractors are liable for the quality of the accomplished works (Enshassi et al., 2008). Therefore, there is a need for an approach, which enables general contractors to measure the quality performances of the candidate subcontractors in an objective, systematic and structured manner, in order to improve the subcontractor selection process.
The main objective of this study is to provide general contractors, especially the ones who predominantly operate in international markets, with a practical and user-friendly performance measurement framework, which assists them in estimating quality performances of candidate subcontractors. The proposed framework is based on the most commonly used KPIs. The proposed framework also aims to enable general contractors: (1) To predict the overall quality performance of the project, (2) To achieve quality requirements specified in the contract and (3) To minimise risks resulting from quality problems.

PrEVIouS STudIES on PErForMAnCE MEASurEMEnT And SuBConTrACTor SELECTIon In THE ConSTruCTIon InduSTrY
The construction sector is criticised by many researchers for its poor performance (Lee, Cooper and Aouad, 2000;Kagioglou, Cooper and Aouad, 2001). Performance measurement in the construction sector has scholarly attracted attention with the increasing complexity of construction projects and developments in construction management and technology (Lin and Shen, 2007;Yang et al., 2010) with studies often focusing on metrics of duration, cost and quality objectives (Ward, Curtis and Chapman, 1991;Kagioglou, Cooper and Aouad, 2001). Recently, new performance indicators such as customer satisfaction, business performance, safety and environment have gained importance (Yu et al., 2007;Yang et al., 2010). Performance measurement has been assessed in three levels: project, organisation and stakeholder.
European Foundation for Quality Management (EFQM), balanced scorecard (BSC) model and key performance indicator (KPI) model are three of the most commonly used frameworks for performance measurement in the construction industry (Yang et al., 2010). EFQM and BSC models are more suitable for strategiclevel performance measurement and company-specific measurement. KPI model, however, is more flexible and can be easily generalised and allows measuring performance at different levels. In this study, the KPI model is selected to construct a framework for measuring quality performances of subcontractor candidates at two different levels, namely project and organisation.
In construction management literature, many studies focused on the subcontractor selection process. Hatush and Skitmore (1997) identified a set of criteria to select eligible subcontractor and proposed a Delphic interview with a small group of construction professionals. Mahdi et al. (2002) proposed a multiobjective decision making technique for selecting appropriate contractors, combining Delphi method with the analytic hierarchy process (AHP). Arslan et al. (2008) used cost, quality, duration and qualification as the main selection criteria in their research, which focused on the subcontractor selection problem. El-Mashaleh (2009) proposed a data envelopment analysis model with 11 criterias for subcontractor selection. Marzouk, El Kherbawy and Khalifa (2013) identified 46 factors that can be used in the subcontractor selection process after conducting a questionnaire survey and carrying out statistical analysis to reveal the most significant factors in the selection process.
There are also studies focusing on performance measurement in the construction industry. Chan and Chan (2004) identified a set of subjective and objective selection criteria to measure project success based on expert opinions. Time, cost and quality were selected as the most important indicators. Luu, Kim and Huynh (2008) proposed a conceptual framework with nine main KPIs to improve project management performance of large contractors. Their findings concluded that the benchmarking approach could help contractors to improve their performances. Enshassi, Mohamed and Abushaban (2009) identified indicators to measure the performance of the local projects by consulting important parties of construction projects, which are owner/employer, consultants and contractor groups. A hybrid Delphi method was used by Yasamis-Seproni, Lee and Arditi (2012) to determine the quality performance of contractors and an extensive literature review was conducted by Ali, Al-Sulaihi and Al-Gahtani (2013) to identify KPIs to measure performances of construction companies at the organisational level in which the most critical KPIs were determined in accordance with relative importance index (RII). Ng and Skitmore (2014) developed a framework to appraise subcontractor using a BSC model and used a case to show the implementation of the proposed framework. Their findings illustrated how the BSC model can help to increase the transparency of the subcontractor appraisal.

METHodoLoGY
The methodology of this study has two phases: (1) a pre-survey and (2) a survey stage (as shown in Figure 1).

Figure 1. Steps of the Proposed Methodology
In the pre-survey stage, firstly, 12 KPIs, which can be used to measure quality performance in construction projects, were identified in the light of an extensive literature review (as shown in Table 1). Then, a questionnaire survey among 40 large scale Turkish contractors, most of which are members of TCA and predominantly act as general contractors, was conducted to determine the relative importance of these KPIs. According to the official figures issued by the Ministry of Labour and Social Security, there are 494 large scale construction companies, which employ more than 250 labourers, in Turkey. From these 494 companies, 100 of them were selected using the purposive sampling method and were contacted. Of the 100 mailed questionnaires, 40 were returned duly filled out, which corresponds to a response rate of 40%. In the questionnaire, the respondent companies were asked to rate the relative importance levels of 12 KPIs using typical five-level Likert scale, where 1 represents the "Least Importance" and 5 represents the "Highest Importance".
Having collected the responses, reliability (i.e. Cronbach's alpha) and ranking (i.e. RII) analyses were carried out in order to test the internal consistency of the scale used in questionnaire for measuring the perceptions of the respondents and to compute the relative importance of the KPIs, respectively.
Cronbach's alpha value was found to be 0.939, which indicates that the internal consistency of the scale used is excellent (Field, 2017). RII is calculated by dividing the arithmetic mean of the responses given for each KPI to the highest importance, which is 5 in this study (Chen, Okudan and Riley, 2010). Based on the findings of the ranking analysis, out of 12 KPIs, six have "High", three have "High-Medium" and three have "Medium" RII. As this study aims to construct an efficient and practical quality performance measurement framework, three KPIs with "Medium" RII, namely availability of quality assurance certificate and date of certificate (KPI#2), subcontractors' management ability of quality assurance and certificate programs (KPI#3) and the establishment of quality assurance system and cost for managing it (KPI#4) were eliminated. The most important nine KPIs with "High" and "High-Medium" RIIs were used when constructing the quality performance measurement framework, namely adequacy of quality assurance policy (Q 1 ), application-level of the quality management system in projects (Q 2 ), the audit of the quality management system in project level (Q 3 ), construction quality (conformance to the specification) (Q 4 ), the quality of workmanship (Q 5 ), the cost of poor quality resulting from the low standard production (Q 6 ), number of reworks (Q 7 ), percentage of reworks (%) (Q 8 ) and severity of defect at construction phase (Q 9 ). The findings of the pre-survey phase were extensively reported in Bingol and Polat (2016).
In the survey stage, the main objective was to calculate the group decision on the weights of the most important nine KPIs (Qs) used in the quality performance measurement framework on the subcontractor selection process. In order to achieve this objective, the AHP method was employed. The target users of the proposed evaluation framework were the large scale Turkish general contractors, who predominantly operate in international markets, are listed in "ENR Top 250 International Contractors" list, are the members of the TCA and had management system certificates (i.e., ISO 9001:2008, ISO 14001:2004, OHSAS 18001:2007. There are 27 Turkish contractors, who fulfilled all these criteria. Therefore, the target population of this research was 27 and all of these companies were contacted. Out of these 27 Turkish contractors, 11 were accepted to contribute to the research. Face-toface interviews were conducted to construct pairwise comparison matrices. The individual judgements of these target users were aggregated to reach a group decision.  AHP was developed by Saaty (1980). It is a basic approach that deals with both the rational and the intuitive decisions to select the most appropriate option from several alternatives with respect to several criteria (Saaty and Vargas, 2012). Fundamentally, the AHP method is based on mathematics and psychology and is the application of an Eigenvalue approach to the pairwise comparisons (Vaidya and Kumar, 2006). The simplest form of AHP is composed of a hierarchy consisting of three levels: (1) The goal of the decision is at the top level, (2) Criteria of the goal are in the second level and (3) Alternatives are located in the third level (Saaty and Vargas, 2012).
The AHP methodology relies on pairwise comparison of either criteria or alternatives. Decision-makers compare criteria/alternative using actual measurements or a fundamental scale to reflect the relative strength of their preferences and feelings. In the pairwise comparison matrices, the question of "How many times the one criterion/alternative is more important or dominant over the other criterion/alternative with respect to the goal/criterion?" should be expressed on a scale of 1 to 9 (as shown in Table 2) (Saaty, 2008). where max m is the principal eigenvalue of the judgement matrix and n refers to the number of criteria in this matrix. The consistencies of decision-makers' evaluation are checked with the consistency ratio (CR). The following equation is used to determine this ratio (as shown in Equation 3).
CR depends on both the CI and the random index. The random index values determined based on different numbers are provided in Table 3. The value of CR for comparison matrices should be less than 0.1 to provide consistency of the decision-maker. If not, the results of the pairwise comparison need to be re-evaluated to improve consistency. In most real-life decision-making problems, decisions are made by several decision makers. AHP is a convenient method to be used in group decision making. There are two different approaches for combining individual judgements, which include: (1) aggregating individual judgements and (2) aggregating individual priorities (Escobar, Aguarón and Moreno-Jiménez, 2004). The weighted geometric mean and the row geometric mean methods are used for combining group decision (Escobar, Aguarón and Moreno-Jiménez, 2004). According to Saaty (2008), taking the geometric mean of individual judgements is the best way to convert individual preferences into a group decision. Usage of 1 to 9 rating scale for individual judgements mainly result in preferring the geometric mean to reach a group decision (Timor, 2011;Onder and Onder, 2014).
When the individual judgements of the decision-makers are combined with the weighted geometric mean method, the final priority is computed via where wij G represents group decision computed with the weighted geometric mean, m represent the number of decision-makers, represents the decision matrix of the decision-maker and λ k represents the weight of decision-maker.

Characteristics of the Interviewed Companies
Out of 27 Turkish contractors, 11 accepted to participate in face-to-face interviews, corresponding to a response rate of 40.74%. Interviews were conducted with 11 civil engineers, who were in charge of subcontractor evaluation and selection process. In these interviews, seven questions were asked to reveal the general characteristics of the interviewed companies. The findings of the general characteristics of the companies were summarised in Table 4. According to the data obtained in the interviews, out of the 11 contractors, 54.54% have 16 to 20 years of experience in the sector while 27.27% have six to 10 years and 18.18% have one to five years of experience. It is observed that 100% of the respondent companies are general contractors where 9.1% of them are subcontractors, 27.3% of them are a member of joint ventures and 9.1% are consultant companies in the undertaken projects. On the other hand, among 11 companies, eight (72.7%) were operating only in international markets and three (27.3%) were operating in both national and international markets. Based on the responses received from the companies, 63.6% of them are specialised in commercial buildings, industrial buildings and infrastructure-transport structures while 45.5% of them are specialised in residential buildings and 18.2% of them are specialised in other types of projects. Findings revealed that 45.5% of the companies sublet 51% to 75% of the total work capacity to the subcontractor companies, 36.4% of them sublet 26% to 50% of the total work capacity, 9.1% of them sublet 1% to 25% of the total work capacity and 76% to 100% of the work capacity to the subcontractor companies.

Group decision on the Weights of the Most Important KPIs (Qs)
Respondents are strategically competitors and the individual preferences of these 11 companies have been taken one by one and then were combined to reach a group decision for the proposed framework. The AHP method was applied to determine the weights of the nine KPIs in the quality performance measurement framework. In the interviews, Saaty's (2008) rating scale was used to calculate the relative importance of these nine KPIs.
In this step, first 11 pairwise comparison matrices were built to understand the individual judgements of each company, the weights of nine KPIs in each company were calculated and the consistency index of each comparison matrix was checked (as shown in Table 5).
Then the individual matrices were aggregated using the weighted geometric mean method into a single matrix to reach a group decision (using Equation 4) and the consistency index of the aggregated pairwise comparison matrix was checked. The aggregated individual priorities are presented in Table 6 and the aggregated weights of nine KPIs are shown in the last column of Table 5.
According to the findings displayed in Table 5, "Construction quality (conformance to the specification)" (Q 4 ) is the most effective indicator with the weight of 0.24009 in the developed framework. This indicator is followed by "The quality of workmanship" (Q 5 ) with the weight of 0.17890, "Application-level of the quality management system in projects" (Q 2 ) with the weight of 0.11874 and "The audit of the quality management system in project-level" (Q 3 ) with the weight of 0.09979. On the other hand, the least effective indicator in the framework is "Number of reworks" (Q 7 ) with the weight of 0.05634.

development of the Quality Performance Measurement Framework
The proposed quality performance measurement framework consists of three steps, which are: (1) inputs, (2) process and (3) output (as shown in Figure 2).    Inputs of the framework include group decision on the weights of the most important KPIs (Qs), which are obtained from AHP calculations, measurement scales for each KPI, the direction of preference of each KPI (minimum is the most favourable or maximum is the most favourable) and the evaluations of all candidate subcontractors with respect to each KPI.
Having determined the inputs' values, in the process step, the performance scores (q) of all candidate subcontractors are calculated by normalising the evaluations concerning each KPI considering the direction of preference. If the direction of preference is maximum, the quantitative equivalence of the evaluation of a subcontractor candidate for any KPI is divided by the sum of the evaluations of all subcontractors to the KPI in question. On the other hand, if the direction of preference is minimum, the reciprocals of the quantitative equivalences of the evaluations are normalised. By this way, the subcontractor candidate with the minimum value becomes more favourable.
In the output step, the overall quality performance index (QPI) of each subcontractor candidate can be calculated using Equation 5. Where QPISC m represents the overall QPI of the mth subcontractor, wi G represents the aggregated weight of the ith KPI (Q i ) based on group decision and q im represents the performance score of the mth subcontractor candidate for the ith KPI (Q i ).

An Illustrative Example of the Application of the Proposed Quality Performance Measurement Framework
An illustrative example has been presented to demonstrate how the proposed framework can be used by a Turkish general contractor, who intends to evaluate the quality performances of the subcontractor candidates under consideration for an international construction project.
In the illustrative example (as shown in Table 7), there are five subcontractor candidates. First, the general contractor determines the measurement scales for each KPI, identifies the direction of preference of each KPI and evaluates all candidate subcontractors for each KPI based on the identified measurement scale. The weights of the KPIs are obtained from the AHP calculations presented in the last column of Table 5. Second, the general contractor calculates the performance scores (q) of all candidate subcontractors. And finally, the general contractor computes the overall QPI of all candidate subcontractors. In the illustrative example, the overall quality performance of the fourth subcontractor candidate (SC_4) is higher than the other candidates.

ConCLuSIon
Since most of the international construction projects are highly complex in nature, they require the involvement of multiple subcontractors; therefore, the selection of the most appropriate subcontractor becomes crucial for the overall project success. The subcontractor selection process is mostly affected by several compromising and conflicting, tangible and intangible criteria. While the criteria like cost and time can be numerically measured, the criteria like safety and quality cannot be easily measured. This study aimed to solve this issue by developing a framework for measuring the quality performances of subcontractor candidates more systematically and rationally way. To use the proposed framework, subcontractor candidates should be assessed based on the identified nine KPIs. After these assessments, the quality performance of the candidates can be determined using the proposed framework. The measurement framework includes indicators that can be measured quantitatively and qualitatively. The limitations of this study are related to the developed framework relying on the individual preferences of the 11 Turkish companies that are also the target users. This potential limitation can be overcome by increasing the sample size. Moreover, the performance of the proposed framework should be tested with real cases.