Relationship between construction performance evaluation and contractor characteristics in Japan

In recent years, in the bidding for public works, it has become increasingly necessary to select a contractor not only by price, but also by the construction quality. This study analyses the relationship between the performance evaluations of construction work on Japanese public works and contractor characteristics. A fixed-effects model and a panel regression model were used for the empirical analysis of data from Keishin, which is a pre-bid qualification review of public works and project performance-evaluation system after the completion of construction in Japan. The results indicate that variables such as years in business and net sales have a significantly positive influence on construction performance. An important aim of national and local governments is to find contractors that carry out high-quality construction, and the results of this study can contribute to resolving this issue. In addition, the results highlight what resources and capabilities contractors should gather to improve their construction quality. Subjects: Construction Economics; Construction Business Management; Business, Management and Accounting


Introduction
Often, the selection of a contractor for public works solely based on price leads to problems with the quality of construction. In addition, in many countries, it is a challenge to identify contractors that perform high-quality construction.
In the United States, the conventional practice is such that only the bid price determines which bid will be granted the contract. However, often, after a successful bid, frequent contract changes become necessary. Especially in large-scale construction, projects are rarely executed at the winning bid price, and the price rises through ordinary contract changes have become normalised. Under such circumstances, the federal government recognised that the competition principle no ABOUT THE AUTHOR Yukiko Konno is an assistant professor at Faculty of Economics, Kokugakuin University. Her research interests include management, financial management and quantitative analysis of company activities.

PUBLIC INTEREST STATEMENT
This study analyses the problem of what kind of construction contractor performs high-quality construction based on actual data. As a result of this study, it is found that construction quality is higher for contractors with larger years in business and net sales. The results of this study are useful for selecting construction contractors for public works.
longer functions, and consequently, a bid criterion named Best Value was designed. In addition to the criterion of price, Best Value considers requirements such as contractor past performance record, technical ability and financial capability (Oono & Harada, 2005).
In the UK, conflicts between contractors and project owners in the construction industry grew intense. Therefore, contractors raising prices repeatedly due to design changes after receiving a low-order bid became commonplace, and many related lawsuits were also filed. For these reasons, the UK government introduced a comprehensive evaluation method that evaluates bids by not only the price, but also by contractor past performance and quality. Furthermore, a so-called framework method was introduced, whereby the project owner also judges contractor quality as a selection criteria (Fujii & Miyakawa, 2016).
According to Olaniran (2015), in Australia, it is feared that poor-quality construction will be carried out if a contractor is selected solely based on price.
Ensuring the quality of Japanese public works has been an ongoing debate. The Act on Promoting Quality Assurance in Public Works (Public Works QA Act) went into effect in 2005, stipulating that "commissioning entities must inspect the technological capabilities of those wishing to participate in bidding". In particular, following the enforcement of this Act, a comprehensive evaluation method has been established that considers not only the bid amounts, but also the quantity of a contractors' orders received and quality of a contractors' previous projects (Konno, 2014). Despite the implementation of these measures, the problem of poor and inferior projects built by incompetent contractors persists, which resulted in the revision of the Public Works QA Act.
The Japanese Ministry of Land, Infrastructure, Transport and Tourism (2011) investigated the relationship between the quality of construction and the successful bit rate (ratio of contractor bid price to the planned price set by the project owner for construction) for construction ordered by this ministry. A statistically clear tendency was found that, for construction projects with a lower successful bid rate, the construction quality is lower. Therefore, it is still an important subject to clarify what kind of contractors will perform high-quality construction.
Prior research has indicated that the proper selection of contractors is critical in ensuring the success of a project because allowing incompetent contractors to participate in the bidding process can lead to defective and inferior project outcomes (Cheung, Wong, Fung, & Coffey, 2008;Hatush & Skitmore, 1997;Ogunsemi & Aje, 2006).
This study aims to analyse the relationship between contractor characteristics and the quality of public works projects. To do so, scores assigned by project owners placing orders, such as national and local governments, to rank the quality of contractors' completed projects are used to link the quality of contractor projects with contractor characteristics. The application of statistical methods to this quantitative data allowed the determination of the relationship between contractor characteristics and the quality of the project.
The hypotheses to be verified in this study are as follows.
H1: Contractors with greater technical specialisation can perform high-quality construction.
H2: Contractors with more experience as prime contractors can perform high-quality construction.
H3: Contractors with more experience regarding time can perform high-quality construction.
H4: Contractors with more experience on a monetary basis can perform high-quality construction.
The study's findings not only identify the characteristics of those contractors capable of executing high-quality projects, but also suggest how these characteristics can be useful as the criteria for the selection of contractors. Moreover, contractors could also use these findings when considering which management resources to develop.
This study is organised as follows: Section 2 reviews the literature. Section 3 explains the system used for evaluating public works project performance (PWPP) in Japan. Section 4 describes the dataset and explains the empirical methodology and hypotheses. Section 5 presents the empirical results and discusses the findings. Section 6 concludes and offers some remarks on future work.

Literature
In this section, this study summaries studies on the relationship between construction performance and characteristics of contractors and on the selection of contractors who undertake construction.
Past studies on the relationship between construction performance and contractor characteristics have determined that construction companies' past performances and technical capabilities have an impact on their construction performance (Alzahrani & Emsley, 2013;Cheung et al., 2008;Khosrowshahi, 1999;Tam & Harris, 1996;Xiao & Proverbs, 2003). Tam and Harris (1996) examined what factors affect contractor performance in Hong Kong and identified the following six factors: (1) project complexity; (2) project leaders' work experience; (3) percentage of professionally qualified staff employed by the contracting firm; (4) contractor's past performance; (5) origin of the contractor and the level of the architect's or client's supervision and (6) control of the quality of work and work progress. Cheung et al. (2008) analysed data on successful bids from the Hong Kong Housing Department, the largest housing provider in Hong Kong and found that past performance score is the most sensitive input variable for the prediction of future performance, followed by price scores.
Many studies on the selection of contractors who undertake construction focused on factors influencing the choice of contractor, with numerous studies determining that a contractor's past performance and preference of the client have an influence on selection (Egemen & Mohamed, 2006;Fong & Choi, 2000;Hatush & Skitmore, 1997;Holt, Olomolaiye, & Harris, 1994;Ogunsemi & Aje, 2006;Watt, Kayis, & Willey, 2009;Wong, 2004). Holt et al. (1994) surveyed 53 major UK construction client organisations and revealed their perceived importance of factors influencing their choice of contractors as follows: (1) contractors' current workload; (2) contractors' experience in terms of size of projects completed; (3) contractors' management resource in terms of formal training regime; (4) time of year and weather; (5) contractor's past experience in terms of catchment, i.e. national or local and (6) experience in terms of type of projects completed. Egemen and Mohamed (2006) examined 91 private building construction clients in Northern Cyprus via face-to-face interviews and analysed clients' needs, wants and expectations from contractor firms. They found that clients in the specialised marketplace emphasised a wide variety of factors; furthermore, clients tend to engage in possible repetitive works with the same contractors assuming that they are entirely satisfied with existing or past projects.
Another study on the process of selecting a contractor pointed out the effectiveness of multiple review processes. Holt (1998) examined construction contractor selection and determined that a two-stage procedure is desirable. Stage 1 includes past performance, experience and financial stability; Stage 2 includes office location concerning the project, experience in the geographical region and experience with the proposed construction methods. Topcu (2004) examined the construction contractor selection model in the Turkish public sector and found that three central concepts are used for selection: cost, time and quality. Topcu (2004) proposed a new selection model that used evaluation criteria related to these concepts and had a process with two main stages: contractor prequalification and the choice of eligible bidder from among prequalified contractors.
On the basis of these prior works, this study clarifies the characteristics of construction companies who perform high-quality construction work. Information on contractors, such as net sales of completed construction contracts and number of engineers, was obtained from the data of Keishin, which is a pre-bid qualification review used by public works in Japan. Furthermore, it is the practice in Japan that after the completion of construction, the national or local government, as the project owner, must evaluate the construction quality. This study uses data on these construction performance evaluation scores and combines them with contractor characteristics. Therefore, unlike questionnaire surveys, this study is empirical, and uses objective numerical data.
Studies on quality construction of public works in Japan have pointed out that it is difficult for contractors to maintain the quality of public works due to issues on the contractor side as well as changes in the Japanese construction industry.
Regarding the project owner side, it is customary that the public works in Japan are paid via a 40% lump-sum payment at the start of construction, with the 60% remaining balance paid at completion. For the above reasons, Kunishima (2001) and Kunishima (2006) pointed out that quality control in connection with the progress of the construction project becomes unclear. Tanaka and Hayashi (2006) argued that ex-post monitoring of the quality of public works in Japan is not done strictly. In particular, from the results of a questionnaire survey, the majority of project owners do not perform ex-post monitoring by third parties.
As a problem facing the construction industry, Furusaka (2012) pointed out that it is difficult to ensure good construction quality due to the recent shrinking size of the construction industry and the unprecedented price competition.
Drawing on studies and considering the current situation of the Japanese construction industry (especially public works), this study clarifies how the characteristics of the contractor affect the quality of construction.

System for evaluating public works project performance
This study empirically analyses the relationship between contractor characteristics and the quality of construction projects in the realm of public works projects. Using the quality of a contractor's projects as a metric, this study evaluates contractors by applying scores assigned by commissioning entities to their completed projects. This section briefly summarises this system.
In an effort to ensure quality of public works projects in Japan, a system for evaluating PWPP was created. This system uses unified standards to evaluate the quality of public works projects executed by contractors. After project completion, the commissioning national or local government evaluates the performance of the contractor that executed the project. This assessment considers various items with respect to the project's execution and construction quality, which are used to calculate a project performance score. The following items are evaluated: (1) construction organisation, (2) construction conditions, (3) appearance and workmanship of the completed project, (4) project characteristics, (5) originality and ingenuity, (6) social considerations and (7) legal compliance. Table 1 shows the specifics and weighting given to each evaluation item. The details of the system for evaluating PWPP have been given by the Ministry of Land, Infrastructure and Transport (2010).
The project performance evaluation scores (PPESs) affect contractors' future bidding for public works projects. The system is structured such that the higher the score, the more a contractor is allowed to participate in bidding for large-scale projects or the greater the preferential treatment they receive, such as being awarded projects at lower bids. Conversely, contractors with very low project performance scores may not be allowed to bid for projects above a certain size or may be excluded from bidding. A perfect PPES is 100 points. Contractors with scores of 80 or higher receive preferential treatment; whereas, often, those with scores below 60 are excluded from the bidding.

Data
Information on contractor characteristics was obtained from Keishin, which is management evaluations of the eligibility criteria for bidding for public works projects in Japan. Details on Keishin have been given by Konno (2014). In addition, scores derived from the PPES are used as a metric for project quality.
This study analyses the relationship between a contractor's characteristics and their average PPES after 1-and 2-year periods to predict how contractor characteristics will affect PPESs after 1and 2-year periods. Due to data limitations, this study uses the average of the scores for the two periods. Namely, the commissioning entities only disclosed an average score per 2-year period.
This study focuses on contractors that have completed at least three general civil engineering projects for the Kanto Regional Development Bureau within a 2-year period. Specifically, this study

Variables and model selection
Among the contractors' characteristics obtained from Keishin, this study used variables reflecting human resources, such as the number of engineers, and capabilities developed from accumulated experience. In particular, the scores from public works performance evaluations are used as explained variables. These included the number of engineers to net sales of completed construction contracts (hereinafter called ratio of engineers), net sales of completed prime construction contracts to net sales of completed construction contracts (hereinafter called ratio of prime construction), years in business and net sales (the unit is one trillion). Table 2 shows the variables used in this analysis and provides descriptive statistics.
The ratio of engineers shows the number of engineers divided by net sales of completed construction contracts. The number of engineers refers to the total number of people with qualifications or education related to general civil engineering projects (e.g. first-class supervisors, first-class engineers, basic technicians, second-class engineers and other engineers). Contractors with higher ratios of engineers on projects have more specialised engineers per unit of net sales of completed construction contracts. Thus, the ratio of engineers is used as a metric to evaluate companies' technical specialisation in construction work.
The net sales of completed construction contracts as prime contractor divided by the net sales of completed construction contracts is used to calculate the ratio of prime construction. In general, contractors with higher ratios of prime construction have more experience as prime contractors for public works projects. Thus, the ratio of prime contractor is used as a metric for evaluating a construction company's experience as a prime contractor.
Years in business indicates the length of time for which a contractor has been in operation. Construction companies with more years in business have more experience as contractors. Thus, years in business serves as a metric to evaluate a construction company's experience regarding time.
Net sales refer to a contractor's total net sales in one year. Net sales include net sales from projects that are not public works projects. In general, as net sales increase, the production output of the construction company also expands; therefore, it can be considered that its experience is rising. It is more accurate to use cumulative net sales. However, due to data constraints, these data cannot be obtained. For that reason, net sales are used as a proxy variable for evaluating a contractor's experience on a monetary basis. In addition, this study uses net sales to control for the impact of company size.

Empirical methodology and hypotheses
This study uses a fixed-effects model in panel regression model for the empirical analysis. Details of fix effect models have been given by Cameron and Trivedi (2005) and Wooldridge (2010). Highly correlated explanatory variables are not included in the model to avoid multiple collinearity.
By designating the ratio of engineers as an explanatory variable, this study can analyse the relationship between human resources and PPESs, and by designating the ratio of prime construction, years in business and net sales as explanatory variables, this study can analyse the relationship between the capabilities derived from experience and PPESs. For the empirical analysis, the four hypotheses presented in Section 1 can be rewritten as follows: H1: Contractors with greater technical specialisation will have higher project evaluation scores.
H2: Contractors with more experience as prime contractors will have higher project evaluation scores.
H3: Contractors with more experience regarding time will have higher project evaluation scores.
H4: Contractors with more experience on a monetary basis will have higher project evaluation scores. Table 3 summarises the results of the empirical analysis. Years in business and net sales have a significant impact on PPESs. Contractors with more years in business and higher net sales have significantly higher PPESs. Therefore, it is found that contractors with more capabilities accumulated through experience regarding monetary value and time length tend to have higher PPESs and, thus, execute higher-quality projects. In other words, hypotheses H3 and H4 were supported.

Results and discussion
Conversely, neither the ratio of engineers nor the ratio of prime construction has a significant impact on PPESs. Therefore, project performance evaluations are not affected by the project performance of contractors with specialised technical resources or by contractors' experience as general contractors. In other words, hypotheses H1 and H2 were not significantly supported.
These findings are in line with those of Tam and Harris (1996) and Alzahrani and Emsley (2013), who examined contractors' past performance and successful projects. However, Tam and Harris (1996) asserted that contractors' project performance significantly improved as their ratio of qualified specialists rose; whereas in this study, the effect of technical specialisation on project performance was not found to be significant. These analytical findings reflect the characteristics of Japan's construction industry. The Ministry of Land, Infrastructure and Transport (2007) indicated that contractors possess unique characteristics not found in other industries. This includes one-off made-to-order production and outdoor mobile production. Moreover, construction is carried out under different individual conditions for each project. Gann (1996) compared the Japanese automobile industry with the industrialised housing industry and noted that unlike the automotive industry, Japan's construction industry has little standardisation of parts. Thus, construction projects have different building characteristics from project to project, making it impossible not only to create replicable work, but also to use such a methodology to check the quality of completed work.
These characteristics of the Japanese construction industry make it extraordinarily difficult to control quality. Therefore, capabilities accumulated through experience, as described by years in business and net sales, have a greater impact on PPESs than technical capabilities and other resources common to the construction industry, such as the ratio of engineers and ratio of prime construction. In addition, it is difficult for contractors to acquire this experience or pretend to have it to gain such capability. Thus, contractors with these capabilities are considered to provide higher-quality construction and, therefore, possess a competitive advantage.

Conclusion
This study analyses the relationship between contractor characteristics and their PPESs to identify the characteristics possessed by contractors that execute high-quality projects.
The results of this empirical analysis show that years in business and net sales have a positive and significant impact on PPESs. In other words, contractors with more years in business and higher net sales tend to have higher PPESs. Conversely, the ratio of engineers and ratio of prime construction have no significant impact on PPESs. Among the four hypotheses presented in Section 1, H3 and H4 were supported: namely, contractors with more experience as measured by the time in business or the monetary size of business are more likely to provide high-quality construction.
However, this study has its limitations, which include an inability to analyse long-term data because subjects of this study are limited to contractors who have executed general civil engineering projects under the purview of the Kanto Regional Development Bureau. Thus, future research will include the analyses of companies working on private projects or in other sectors in addition to the contractors in this sample, which would allow the use of longer-term panel data for examining the impact of industry characteristics or the macro environment. In addition, because this study cannot use control variables because of data constraints, future research will create an original database and use more explanatory variables. Cheung et al. (2008) indicated that past project performance scores are useful for predicting future performance. Therefore, a future study will analyse the relationship between past and future project performance.
The contribution of this study is to analyse the relationship between contractor characteristics and performance ability on construction quality based on objective numerical values contained in the data from Keishin and the construction performance evaluations. This study provides useful suggestions for discussions of the criteria for selecting contractors as well as business strategies, namely, that contractors should acquire management resources.

Funding
The author received no direct funding for this research.