Efficiency measurement in the tendering of road surface renewal contracts

Abstract

The problem of slow productivity growth in the road construction (and wider construction) industry is well known. The present paper suggests a means for efficiency analysis in one part of this industry, namely road surface renewal in Sweden, built upon the application of Stochastic Frontier Analysis (SFA) techniques. The paper is novel in that it focuses on project level rather than firm or contractor level performance and takes the perspective of the inefficiency that may result from the way the contracts are specified by the highway agency’s pavement engineers (client side). We compare 233 renewal contracts tendered over a four-year period via the estimation of a cost frontier, with controls for heterogeneity between projects. Our results produce first estimates that expose substantive differences in the relative efficiency performance of different engineers within the Swedish highways procurement organisation (Trafikverket); with indicative savings of around €40 m out of a total road renewals budget in Sweden of €200 m. We also find substantial economies of scale that could, in principle, point to further cost savings if road renewal projects can be packaged up as larger projects. These client-side savings represent potentially important sources of savings in addition to those that can be achieved through the pressure of competitive tendering on the supplier side. The paper therefore illustrates how disaggregate analysis of project level information can readily be used for revealing important information about how best to frame the procurement process and thus deliver productivity and unit cost improvements over time.

1 Introduction

In most countries, governments are responsible for providing road infrastructure services, including the construction of new roads as well as the renovation and maintenance of the existing network. One motive for studying cost efficiency issues in the provision of these services is the substantial sums of public money spent. Cost efficiency is concerned with the transformation of inputs (costs) into output; in our case, the cost measure is the total cost of each single project, and the output is a measure of the volume of road re-surfaced (these aspects to be defined further below).

Out of a total central government budget of EUR 90 billion, EUR 2 billion was earmarked for road infrastructure renewal and another EUR 2 billion for infrastructure maintenance in Sweden in 2016.¹ The latter budget item inter alia includes renewal activities whereof pavement resurfacing is allocated about EUR 200 million (Proposition 2017/18:1 2017).

Except for the possibility to reduce costs, a further motive for investigating the cost-efficiency performance of this sector of the economy is related to the fact that productivity growth in the European construction industry more widely has been relatively poor. This statement is based on national accounts data from different European countries collected in the KLEMS² database (see, Jäger 2017; The Economist 2019; Salomonsson et al. 2019).

Our contribution to the analysis of cost efficiency is based on micro, project level data, rather than the aggregate analysis of data from national accounts or KLEMS. Specifically, we analyse the cost efficiency of one specific type of activity, namely the recurrent road reinvestment activities referred to as pavement resurfacing which concerns the process of replacing one or more new layer(s) of asphalt over the existing pavement.³

The Swedish Transport Administration (subsequently referred to as Trafikverket or the principal) is responsible for national roads. Rather than using in-house resources, all road construction, renewal and maintenance activities are competitively tendered. One obvious consequence is that the way in which the procurement process and the subsequent contract with a builder is designed is important for the efficient functioning of this segment of the construction industry.

For this study, documentation of costs for, and properties of 233 pavement renewal contracts tendered between 2012 and 2015 has been compiled. All of these renewals are for projects with hot mix asphalt, i.e. the type of asphalt typically used on roads with much traffic. By limiting the study to one type of asphalt, the technology used for all contracts in the sample is very similar.

Our focus is not on the efficiency of the surfacing companies, as we assume that the competitive process eliminates major inefficiencies across contractors. Starting with the early work of Demsetz (1968), the theoretical literature suggests that ex-ante competition (competition for the market) should lead firms to operate efficiently (at least where there is a healthy competition and no information asymmetries). The transport and wider procurement literature provides empirical support for this notion with competitive tendering typically producing substantial efficiency improvements across a range of sectors (see Section 2); though the literature recognises that there may factors that limit the ability to realise these gains. Here we note that the mean (and median) number of submitted bids in our data is 4 which indicates strong competition for road surface renewal contracts in Sweden.

The focus of this paper is therefore rather on the way in which each assignment is specified by the client side in a Quote for Bids. This document is an invitation to suppliers to submit bids. The Quote initiates the standard business process and results in rewarding the preferred bidder with each contract.

Each region within Trafikverket has a team that specifies which roads to prioritise. The operational responsibility for tendering surface renewal is with Trafikverket’s pavement engineers. Once the roads that require renewal have been identified, engineers design the Quote for Bids for each project which sets out precisely what the contractor is supposed to do. Contracts are allocated based on the least cost bid.

Our focus is on the way in which the pavement engineers specify each assignment in the Quote for Bids. Our dataset comprises two or more engineers in each of the principal’s six regions. This implies a form of “sub-company” panel structure (see Smith et al. 2012), with multiple observations (contracts) that each engineer has been responsible for procuring. Using these repeat observations on the same engineer – the various projects that they procure – provides estimates of inefficiencies resulting from the performance of each engineer relative to their peers. The key assumption for the analysis is that for each road re-surfacing project of a given size, the pavement engineer has discretion over the content of a Quote for Bids and the subsequent contract.

The paper uses a cost function that controls for the unique preconditions of each project.⁴ This includes the road quality before treatment and other characteristics that are exogenous from the viewpoint of the engineers. The analytical strategy is to eliminate differences in external fundamentals and to attribute remaining, unexplained variation in contract costs to the skill of the procurement engineers.

The cost function used for the analysis assumes that the project size (m² of pavement) is exogenously given. With this number as a starting point, the engineers specify the activities that they consider to be necessary for delivering the new road surface. The source of engineer-side efficiency can therefore result from the fact that “inefficient” engineers over-specify the activities that are needed to deliver a given level of output.

There is a degree of ambiguity in the analysis in that the engineers may exert some control over contract size itself. We seek to control for this by applying panel data methodologies that allow for correlation between the engineer level effects and the regressors in the model. Therefore, in addition to engineer (client-side) inefficiencies, our analysis permits the investigation of the extent of economies of project size, and the potential cost savings that could be achieved through procuring road renewals as larger projects. We use Stochastic Frontier Analysis (SFA) as a means for efficiency analysis of this procurement process.

The available information refers to winning bids and not the actual, outturn cost of the projects. It is well known that costs ex-ante often differ from costs ex-post, but this is only a problem for our efficiency analysis if such deviations are systematically different between the analysed engineers. This robustness is a beneficial consequence of analysing relative efficiency, where the internal comparison is not affected by factors which are about the same, on average, to all analysed individuals. In our case, by analysing the relative efficiency of engineers specifying only one type of road work, the relative efficiency measurements are robust to the general propensity and magnitude of ex-post changes for that type of work.

One source of ex-post cost changes is called unbalanced bidding. The literature has established that there may be reason for bidders to distort the (unit) prices of the activities that they find likely to be under- or over-specified by the procurer, maximizing the expected profit based on their expectations on the final quantities. However, following the line of reasoning above about robustness, the existence of such strategies only affects the validity of relative efficiency measures based on winning bids if the engineers are systematically different in terms of unbalanced bidding. Regarding the extent of this source of ex post cost change, empirical studies of US data find evidence only of small magnitudes of unbalanced bidding (Bajari et al. 2014; Miller, 2014) and Nyström & Wikström (2019) finds no evidence of such strategies in Swedish investment projects.

The non-availability of ex-post project costs does not admit drawing firm conclusions in respect of relative efficiency as noted. Since our purpose is develop a generic approach for utilizing project data using the well-established SFA method, rather than to assess the engineers’ relative efficiency performance in this particular dataset, this is less of a concern. We, therefore, consider the results to be first estimates of the potential savings possible from addressing potential client-side inefficiencies.

Any research in this field must have access to pertinent empirical data, which is notoriously difficult to acquire. Our analysis and the unique project-level data set on which it is based, therefore offers a rare opportunity to address the relative efficiency performance of the client-side engineers that are instrumental for the way in which the process is executed. A better understanding of performance differences will make it feasible to identify particularly good performance which provides a starting point for a subsequent discussion of best practice and for the gradual reduction of unit costs and productivity improvement. Our paper is therefore also relevant to a wider, largely US-based literature working with micro information about projects to learn about bidder behaviour and the way the construction industry operates (see for example, Bajari et al. (2014)). The possibility to dig behind the surface of project-level information by compiling the detailed information about contracts has a potential for providing a much better understanding on the tendering and implementation process in construction.

The paper is structured as follows. Section 2 sets out a summary of the literature. Section 3 explains the conceptual framework for the analysis. Section 4 sets out the data and the empirical model is outlined in Section 5. Section 6 presents the results and the conclusions are given in Section 7.

2 Summary of the relevant literature

There is an extensive literature studying the marginal cost of using road infrastructure. This is motivated by the fact that an efficient and sustainable use of transport infrastructure presumes pricing its usage based on the marginal cost of road use (Nash and Sansom, 2001; Nash and Mathews, 2005; Link, 2014). A second, related literature uses data from national accounts. This research was given a boost when The European Commission funded the compilation of industry level, comprehensive and harmonised national accounting data. The data is open source (see O’Mahony and Timmer 2009).

However, the latter type of analysis considers productivity at the aggregate, industry sector level. Considering the large amounts of resources expended on road construction and maintenance, the empirical literature analysing bidding behaviour as a function of the bidding framework that is established in the Quote of Bids is relatively small. Examples include Silva et al. (2008), empirically testing the hypothesis that the release of information regarding the seller’s valuation of an item in the Quote can result in more aggressive bidding and Li and Zheng (2009) who compare three competing procurement auction models with endogenous entry. See also Bajari et al. (2014) referenced elsewhere in the paper.

Although the idea of benchmarking efficiency has a long history in the academic literature in general, the availability of econometric methods gives renewed impetus to more advanced research (Nash, 2018) in the road sector. One example is Welde and Odeck (2011), who study the efficiency of 20 Norwegian toll companies in operation between 2003–2008 using data envelopment analysis (DEA) and stochastic frontier analysis (SFA). The findings seem to indicate a potential for efficiency improvement, but the variation in the efficiency scores is dependent on the method used. See also Odeck (2014).

The adoption of cost-efficiency measurement approaches provides valuable recommendations on how to contract out road maintenance in an efficient manner. Wheat (2017) analyzes the efficiency in road maintenance for 51 local authorities in England and finds that sharing maintenance services (or mergers) across small local authorities would lead to potential cost savings. Moreover, the paper highlights the scope for 17% cost savings without compromising maintenance quality and level of traffic flow. Previous work has also emphasized that the type of the contract is an important determinant of the efficiency of road maintenance (Fallah-Fini et al. 2012), as well, for example, the duration of the contract (Anastasopoulos et al. 2010). Contract size also has an impact. For instance, Link (2006) suggests tendering larger lot sizes due to the existence of economies of scale in highway renewal. Wheat et al. (2018) consider efficiency of the procurement process with focus on the presence of outliers in the data and its implications for stochastic frontier analysis.

Yarmukhamedov et al. (2020) use econometric techniques to study efficiency variation in 73 tendered contracts for road maintenance over an 11-year period. Approximately two-thirds of the costs in these contracts are used for winter maintenance and the rest for minor pavement repairs, cleaning and grass-cutting, etc. They note the international evidence that competitive tendering has tended to deliver large savings in road maintenance costs in a range of countries. This finding has also been observed in many industries in many countries around the world (see for example, Domberger et al. 1986, 1987; and Alexandersson, 2009). The argument is that ex-ante competition (competitive tendering), disciplines firms to improve efficiency (Demsetz, 1968).

Our study contributes to the above literature, but our interest is not on the efficiency of the surfacing companies, but rather on the way in which procurement engineers specify the task in the Quote for Bids that initiates the tendering process and in the subsequent contract with the assigned winner. We focus on micro, project (contract) level observations rather than the aggregate analysis of data from national accounts and from KLEMS. Specifically, we analyse the cost efficiency of one specific type of activity, namely the recurrent road reinvestment activities referred to as resurfacing. Our analysis and the unique data set on which it is based, therefore offers a rare opportunity to understand the relative efficiency performance of the officials that are instrumental for the way in which the process is executed.

3 Conceptual considerations

Taking as given the characteristics that are exogenous for roads in the six regions, the engineers that are responsible for tendering contracts may fail in identifying the cost-minimising activities and quantities for each project in the Quote for Bids. Since we are unaware of what these cost-minimising quantities are, it is necessary to formulate a conceptual framework for using observed prices and quantities to derive efficiency differentials between engineers.

To do so, it is essential to formalize the structure of the contracts that are used. Tendering of pavement renewal is based on the Unit Price Contract (UPC) framework where a Bill of Quantities is a core part of the Quote for Bids that triggers the submission of bids from road surfacing contractors (cf. Bajari et al. 2014). The Bill is a quantified list of all activities to be implemented by the winning contractor. A bid comprises a unit price for each activity, and the aggregate bid is the product of price and quantity vectors. Bidders compete for contracts by holding down their (unit) cost for each activity.

To be precise, the cost (winning bid, C) of project ρ is the quantity (Q^ρ) of l = 1…L activities, $Q_{li}^\rho$, specified in the contract by engineer i times the price, $P_l^\rho$, set by the winning firm⁵.

$$C_i^\rho = \mathop {\sum}\limits_{l = 1}^L {P_l^\rho Q_{li}^\rho }$$

(1)

In the same way, the minimum cost is defined by Eq. (2). This cost is not influenced by which specific bidder that wins or which engineer that has prepared the quote, but rather by the properties of the road and actual prices of materials and labour.

$$\widehat C^\rho = \mathop {\sum}\limits_{m = 1}^M {\hat P_m^\rho \hat Q_m^\rho }$$

(2)

In optimum, m = 1, …, M tasks are to be performed. The number of cost-minimizing tasks can be larger, equal or less than those specified in the contract. However, by letting tasks that may be specified in the contract, but would not be included in the optimal contract, have value zero in optimum, M ≥ L.

Both the engineer and the firm may cause inefficiencies such that the final outturn cost of the project will be higher than $\widehat C^\rho$. The engineer may specify too many or too few tasks or may have an estimated quantity that differs from optimum while the bidder may have market power or be inefficient, resulting in unit prices above the optimum. To account for these possibilities, the actual and the optimal cost are linked by adding two inefficiency terms where u_mf is a deviation from the optimal cost due to non-optimal behaviour of the firm concerning task m. Similarly, u_mi is a deviation from the optimal cost due to the engineer in relation to activity m. The total optimal cost is assumed to be lower than the bid cost, and the two sums of deviations are non-negative, u_f ≥ 0, u_i ≥ 0. Thus, u_i can be interpreted as inefficiency due to engineer i. The focus of this study is to estimate this term for all the engineers; as noted in the introduction, we assume that the competitive tendering process handles firm level inefficiency.

$$C_i^\rho = \mathop {\sum}\limits_{l = 1}^L {P_l^\rho Q_{li}^\rho } = \hat C^\rho + \mathop {\sum}\limits_{m = 1}^M {u_{mf}} + \mathop {\sum}\limits_{m = 1}^M {u_{mi}} = \hat C^\rho + u_f + u_i$$

(3)

Only bid costs, not actual costs are observed, meaning that the former is used as a proxy for the latter in the model. Since the optimal number of tasks and the volume of each could be larger than those specified by the engineer, the winning bid-cost could be lower than the optimal cost. That is, an engineer could specify the project incorrectly in such a way that too little activity is specified. In that situation, the failure to specify the optimal quantities at the start would eventually lead to an outturn actual cost that is above the optimum. Such deviations are mainly only a problem for our efficiency analysis if they are systematically different between the analysed engineers, as efficiency will be assessed relative to the most efficient engineer rather than in absolute monetary terms.

Linking this conceptual framework to the output per input definition of efficiency, we model the cost input (bid costs) as a function of output plus a set of exogenous characteristics of that output. The model thus captures heterogeneity between projects that the engineer must relate to, i.e., that are exogenous to the engineer, and should therefore not be correlated with engineer efficiency. Deviations from the modelled cost relationship can then be interpreted as a combination of engineer inefficiency and random noise (see Section 5).

4 Data and aggregation

The data set has been compiled within our project with the support and assistance of Trafikverket staff. Documentation of costs for and properties of 233 pavement renewal contracts tendered between 2012 and 2015 has been compiled. All of these renewals are with hot mix asphalt, i.e. the type of asphalt typically used on roads with much traffic. This asphalt is prepared at higher temperatures (>120 °C) than other mixes (e.g. warm and cold mix), and must be prepared at an asphalt plant and be transported and applied before cooling below some critical threshold. The other types of asphalt may be prepared at the site. By limiting the study to one type of asphalt, the technology used for all contracts in the sample is very similar. Sweden does not have concrete pavements.

The purpose of Section 4.1 is to further describe the preconditions of the pavement engineers entrusted to specify the Quote for Bids. This provides the starting point for describing the data that is used in Section 4.2.

4.1 What is exogenous and endogenous for the pavement engineers?

Rewarding a contract to a company is the final step of a chain of decisions within Trafikverket. The first step means that the principal’s central level (its main office), distributes the resources for pavement renewal – €200 m mentioned in the introduction – over the six organisational regions. The size of the regions’ allocation is based on annually updated observations of road quality relative to set standard targets in the different parts of the country.

The choice of road sections to be upgraded is made by a group of officials at each region. Bad road quality is at the core of this choice. To establish quality, laser vehicles scan roads on a regular basis. Two statistics are used for identifying road quality, rutting (mm rut depth) and longitudinal unevenness measured by the International Roughness Index (IRI). The trigger values are not homogenous across all roads. The basic principle is that the more vehicles that use a road and the higher the speed limits, the higher is the requirement for road smoothness.

The pavement engineers are responsible for transferring each road section that is shortlisted into a Quote for Bids and subsequently to a contract with the low-cost bidder. This work is done within a structure established by the organisation’s national level. Guidelines are based on a life cycle cost perspective, i.e., to apply materials and methods that provide lowest possible costs over the expected life cycle of the pavement. The central level also designs of the procurement process which is set into actual use by the pavement engineer.

Within this framework, engineers have substantial degrees of freedom. Even within the hot mix category, there is a choice between different types of asphalt with respect to different combinations of stone size etc. Another basic choice concerns the combination of road sections that are shortlisted into comprehensive contracts. Two or more sections may have below-standard quality while there are sections between them with acceptable standard. It may be cost minimising to improve several sections – up to several kilometres – at the same time to avoid the necessity to have to come back and handle the in-between sections after another few years. Many contracts moreover include several longer sections without direct physical contact but that are geographically not far away from each other. Combining sections into one contract makes it possible to use the same heavy laying equipment without having to move it longer distances, i.e., again a way to save on costs.

The Quote for Bids for a project is our basic object of observation. The start and end of a road section provides information about the area to be treated (road length times width). By specifying the thickness of the asphalt layer and which asphalt blend to be used, the project can interchangeably be quantified by area (m²), asphalt volume (m³) or weight (since each blend has a specific weight, ton).

Except for establishing project size (m²), the Bill of Quantities typically include activities that are complementary to the core task. This can comprise actions necessary for preparing a road before the new asphalt could be spread (for instance by removal of the existing surface to avoid the road level becoming too high) and the specification of activities for finalization of an assignment (adding a gravel string at the edges of the road after that the pavement has been applied). The Bill may also establish that more than one asphalt layer is to be spread by the surfacing contractor. More than one layer will, ceteris paribus, enhance the expected life of the surface.

This description establishes which type of choices that must be made by the pavement engineers. The statistics that establish which roads to improve may also affect the treatment costs. One reason is that the worse the initial quality, the more costly may the treatment be. In addition, more traffic makes it more costly to undertake the works. Moreover, both speed restrictions and roads passing through conurbations may increase costs because of more challenging preconditions for the works.

Project costs may also be affected by the closeness to the surfacing contractors’ plant: the shorter the distance between the plant used for mixing crushed stone with bitumen and the project, the higher is the chance that the owner of that plant has a competitive advantage over other bidders. This aspect is addressed in Ridderstedt and Nilsson (2023).

The next section provides a description of the information available about these external variables. To repeat, the key assumption for the analysis is that for each road re-surfacing project of a given size, the pavement engineer has discretion over the content of a Quote for Bids and the subsequent contract. All features of the tendered contracts that are described in the Quote for Bids are therefore assumed to be endogenous and contribute to whether an engineer is more successful than the peers.

4.2 Data

Table 1 enumerates external project characteristics from four different sources (see the notes to the table). The first source gives information on the cost and size of the projects. The first variable in the table is the dependent variable, project cost, which ranges from €100 000 to €4 890 000 (see Table 2). This variable corresponds to the total cost of the inputs specified by the engineer. The second variable is the total surface area in each project, ranging from 11000 to 632000 m². This is our measure of output, assumed to be pre-decided at the time when the engineer specifies the project. The subsequent analysis thoroughly tests the significance of this assumption for the results.

Table 1 List of variables

Table 2 Summary statistics for 233 projects

For improving comparability, we have collected data on potentially cost-influencing characteristics of our output variable. From the second source (the national roads database), information about the number of vehicles on each road, measured as average daily traffic (ADT), i.e., the yearly average of number of vehicles passing during a 24 h period, is compiled. The average is some 6000 vehicles (see Table 2) including roads from 500 up to 55,000 vehicles per average day. In the same way, speed limits range from about 40 to 110 km/h. This source also provides information of whether the road sections included in a renewal project are located in an urban area. A road section passing across the countryside has the value 1, while the value is 2 when the road passes a built-up area. Typically, a renewal project includes several road sections, and the variable (Urban) is the average value of all road sections included in the project (which means this is a continuous variable, taking values between 1 and 2). Since the average value of this variable is about 1.1, Table 2 establishes that most projects in our sample mainly cover road section outside built-up areas. This can be expected, as the contracts are for renewal with the durable hot-mix asphalt often used on highways, and procured by the national IM rather than the municipalities responsible for most of the road network in the cities.

A third source comprises information about the road surface quality statistics prior to tendering the contracts, i.e., IRI and track depth. Finally, a measure of the number firms with at least one asphalt plants localised in a radius of 150 km from the project site has been collected (AP150, source 4). This provides a control for cost differentials which may emanate from the degree of (potential) competition in different parts of the country.⁶ A complementary measure with the same background is the distance (Distance) between the plant and the work site for the winning surfacing contractor.

Significant additional information is available about the variables specified in the Bill of Quantities. Since these design features of the contracts are assumed to be under the control of the engineers they are not included as explanatory variables in the analysis. This point is central to the analysis. Whilst it would in principle be possible to explain project cost variation in terms of differences in a large range of quantities of materials included in the Bill of Quantities, these can be considered inputs under the control of the engineers. Instead, we characterise heterogeneity in projects, as is standard in cost function analysis, by reference to exogenous characteristics such as traffic volumes, road quality and speed capabilities. We then seek to measure efficiency having controlled for these characteristics.

The national road network comprises more than 400,000 homogenous road segments. A new segment is defined as soon as a vital quality of the road changes. ‘Vital’ may refer to road width, a road crossing (where there may be a change in the number of vehicles), a bridge etc. Sections vary between a few meters in length up to more than 1500 meters. All contracts comprise several road sections. Since the road quality parameters (IRI and track depth) are registered at the road section level, this information is aggregated up to the project level in the following way. Using observations from the measurement closest to the start of each project, an aggregate measure of quality on the project level is calculated as:

$$Quality_p = Wmean_p\left( {Med_1, \ldots ,Med_j, \ldots ,Med_{K_p}} \right)$$

Med_j is the median of the quality measure of road segment j and Wmean_p is the weighted average over all median values for segments in a project p. The weights are based on the length of a road segment divided by the total length of the segments in the Bill of Quantities, meaning that median values of long segments are given larger weight. The median is used due to the noisy nature of the quality measures.

Table 3 summarize administrative information about the 233 projects that have been tendered over 4 years and by six administrative regions. Seven firms who hold between 2 and 48 contracts have been contracted. For each contract, one responsible engineer is named in each contract’s Bill of Quantities. In total, 24 engineers are named, subsequently numbered from 1 to 24. The number of projects per engineer ranges from 4 to 24. Trafikverket publishes an index for road maintenance costs. Since this index has not changed during these four years, the original cost observations have not been adjusted for inflation – but given that the index has not changed over the sample, the cost data can be considered to be expressed in real terms.

Table 3 Administrative description of projects

5 Empirical model

Our general framework for structuring the efficiency analysis is to compare differences in the total costs of the inputs specified by an engineer to achieve a given output (the surface area of the road segment that is to be replaced), controlling for exogenous cost-influencing differences in the characteristics of that output (e.g., traffic density, speed, road degradation). The flexible trans-log function is employed as is standard in the railway and wider cost efficiency literature (see e.g., Coelli et al. 2005):

$$\begin{array}{l}lnC_{rip} = \alpha _{ri} + \mathop {\sum}\limits_{k = 1}^K {\beta _kX_{krip}} + \mathop {\sum}\limits_{k = 1}^{K - 1} {\mathop {\sum}\limits_{j > k}^K {\beta _{kj}X_{krip}X_{jrip}} } \\\qquad\quad\;\;\;+ \mathop {\sum}\limits_{k = 1}^K {\beta _{kk}X_{krip}^2} + \mathop {\sum}\limits_{m = 1}^M {\gamma _mZ_{mrip}} + \varepsilon _{rip} = \alpha _{ri}\\\qquad\quad\;\;\;+\, {\Bbb X}_{rip}B + \varepsilon _{rip}\end{array}$$

(4)

lnC_rip is the log-transformed total cost for project p in region r procured by engineer i. X_krp represent the log-transformed variables discussed in Section 4, variables Z_mrtp, m = 1,…, M, are dummy variables capturing region and year effects, α_ri is a unique effect for engineer i in region r and ε_rip, is a random error. The model is summarised in Eq. (4) with the row vector ${\Bbb X}_{rip}$ including all the right-hand side proxy variables and B is a column vector including all variable coefficients.

Both random and fixed effects versions of our models are estimated. For the fixed effects estimation, the relationship between the inefficiency and the regressors in the model is unrestricted. For the random effects, the assumption is that the engineers’ inefficiency is independent or uncorrelated with the regressors. We return to discuss the positioning of this approach within the broader efficiency estimation literature below after setting out the model more formally.

To re-iterate, our explanatory variables are – by construction – exogenous from the perspective of the engineers. However, one challenge for the analysis is whether our output variable, i.e., the size of the contracts (m² asphalt), could be considered to be partly under the influence of the engineers. If this size variable hides inefficiency, we conjecture that it will be correlated with the remaining inefficiency. For example, if the engineer predominantly sets square meter specifications that lead to unnecessary costs, then the engineer may also make other misspecifications in the contracts that causes extra cost. Given this conjecture, a Hausman test of random effects is computed (both in general, and in connection to the project size variable).

For random effects estimation, we rewrite Eq. (4) as follows:

$$\begin{array}{l}lnC_{rip} = \alpha + E\left( {u_i} \right) + \alpha _r + {\Bbb X}_{rip}B + \varepsilon _{rip} + u_i - E\left( {u_i} \right)\\\qquad\quad = \alpha ^ \ast + \alpha _r + {\Bbb X}_{rip}B + \varepsilon _{rip} + u_i^ \ast \end{array}$$

(5)

In Eq. (5), the engineer specific effect is separated into three parts as follows: α_ri = α+αr+ui; that is, a constant term, a region-specific effect, and a term picking up engineer efficiency. In this case the inefficiency term is assumed to be uncorrelated with the explanatory variables including regional dummies that are used to estimate α_r. Using tilde to denote the random effects estimators, the first step is given by Eq. (6):

$$\widetilde u_i^ \ast = ln\overline C _{ri} - \widetilde \alpha ^ \ast - \widetilde \alpha _r - \overline {\Bbb X} _{ri}\,\widetilde B$$

(6)

and secondly the following difference is taken:

$$\widetilde u_i = \widetilde u_i^ \ast - \mathop {{\min }}\limits_{{{\mathrm{j}}}} \tilde u_i^ \ast$$

(7)

Efficiency scores (ES) are by construction between zero and one and defined as follows:

$$ES_i = \frac{{E\left( {\left. {C_{rip}} \right|{\Bbb X}_{rip},\alpha _r,u_i = 0} \right)}}{{E\left( {\left. {C_{rip}} \right|{\Bbb X}_{rip},\alpha _r,u_i} \right)}} = \frac{{e^{\alpha + \alpha _r + {\Bbb X}_{rip}{\rm B}}E\left( {e^{\varepsilon _{rip}}} \right)}}{{e^{\alpha + \alpha _r + u_i + {\Bbb X}_{rip}{\rm B}}E\left( {e^{\varepsilon _{rip}}} \right)}} = e^{ - u_i}$$

(8)

The efficiency score is the number that the conditional average cost of one engineer should be multiplied by to obtain the same kind of average for a completely efficient engineer. If, for instance, ES_i = 0.8, then an efficient engineer could do the same kind of project for 80% of the cost, on average.

The econometric model is a type of stochastic frontier, albeit one estimated without distributional assumptions (see for example Schmidt and Sickles 1984). One alternative would be to estimate inefficiency by deterministic optimization methods, like DEA. However, DEA does not handle random variation and our use of proxy-variables to capture differences between projects inevitably implies measurement errors that DEA has no means to handle due to the deterministic nature of the method. A benefit with DEA is that it imposes monotonicity, i.e. that output may be constant but never decreasing with respect to the inputs, in line with microeconomic theory on production functions. When applying stochastic frontier models, it is important to check whether this property is fulfilled (O’Donnell and Coelli 2005; Sauer et al. 2006; Henningsen and Henning, 2009). Within our empirical framework, this concerns the association between the input variable, the winning bid, and the output variable, “m²”. If all the observation-specific elasticities for “m2” are non-negative, we consider monotonicity to be fulfilled.

More complex stochastic frontier models are also available. For example, the so-called “true” fixed and random effects models (Greene 2005a, 2005b) are designed to capture unobserved heterogeneity. Here it should be noted that the most common panel data set-up observes firms over time. However, Smith et al. (2012) considers sub-companies within firms, such that multiple observations on the same firm – different parts of the same firm – make up the panel structure. Our approach is akin to this approach in that the panel is defined by multiple observations on the same engineer through the different projects they are responsible for. This enables us to obtain engineer-level estimates of inefficiency.

The disadvantage of the more complex models is that strong assumptions are made about the population distribution of efficiency terms and random errors, for example that inefficiency of the same unit develops completely randomly over time. Translated to our setting, the inefficiencies would then be independent between projects handled by the same engineer. The current approach rather assumes engineer efficiency to be constant across projects. Indeed, one feature of our model is that the inefficiency invariance assumption – namely that all projects organised by the same engineer would have the same inefficiency – is a more reasonable assumption than the traditional panel model, where time invariant inefficiency may generally be quite restrictive.

6 Results

The results from two different models representing the Translog function are presented in Table 4. Model 1 is a fixed effect specification that regresses (log) of bid costs on the project characteristic variables described in the data section, including year and firm dummies. As noted, the engineer is the panel identifier, with multiple project and time observations for each engineer. Model 2 is the random effects version of Model 1.⁷

Table 4 Regression results (Trans-log)

For brevity and clarity, we show the elasticity evaluated at the sample mean for each variable, as it is difficult to interpret the individual estimates of the first- and second-order terms and interactions. The elasticities give the percentage change in cost associated with a 1% increase in the variable. The full Translog results, covering the estimates of each term and the standard errors (clustered on the engineer), are provided in the Appendix. We carry out joint testing, considering first- and second-order terms of each variable. According to the joint significance test, all but two of the variables are significant in both models. The remaining two variables, AP150 and TrackDepth, are close to significant at the 95% level.⁸

The sample mean elasticities of the control variables are similar with both models. The elasticity for m² implies that, at the sample mean, cost increases by around 0.72–0.73% if the scale of the project is increased with 1%, i.e. the results suggest economies of scale at the sample mean. Contrary to our expectation, the elasticity of distance is negative, implying that the cost reduces with the distance to the winning bidder’s asphalt plant. Possibly, this variable picks up an effect of other cost influencing factors associated with the distance of the winning bidder, for instance differences in input prices between regions. In Model 2, regional differences are controlled for and the mean elasticity of distance is half the magnitude in Model 1. The remaining control variables have the expected sign: the elasticities at the sample mean suggest that cost is higher for pavement renewal projects for roads with higher traffic flows (ADT), worse initial condition of the road surface (IRI), more segments in urban areas (Urban), and higher speeds (Speed). Also the variables that are not statistically significant have the expected sign, i.e. that cost decreases with many potential bidders (AP150) and increases with our second measure of the deterioration of the road surface (TrackDepth).

We have performed Hausman tests, comparing Model 1 and 2. If the size of projects variable (measured by m²) hides inefficiency, the previous discussion has established that the variables and the remaining inefficiency of the model should be correlated. This would be a violation of the RE model. However, the Hausman test gives no evidence of such a violation. The p-value is 0.65, when the test is only based of coefficients on terms including m² (the variable that we consider most likely to present possible endogeneity issues). Also with a full Hausman test based on a comparison of all coefficients in Model 1 and 2, the RE model is not rejected. The p-value is 0.99. Thus, we find no evidence that the variables from the UPCs hide inefficiency of the engineers, meaning that random effects estimation is preferable. Lastly, we find that the property of monotonicity is fulfilled with both models, with all observation-specific elasticities of m² being positive (within the range of 0.31 and 1.07).

Figure 1 provides the efficiency score estimates for each of the 24 engineers, first for the FE and thereafter for the RE estimates. The correlation between the two models’ efficiency estimates is 0.95 and most efficiency scores above (below) the average for the RE model are also above (below) the average score in the FE model. The average efficiency score for RE is 0.82 and the FE is 0.73. Taken at face value, this finding implies that if all engineers matched the efficiency performance of the best, total costs of road resurfacing in Sweden could be reduced by 18–27%. Since around EUR 200 million is spent on surface renewal, and interpreting the 18–27% range cautiously, this corresponds to a saving of around 20% or EUR 40 million per year. The variation in efficiency between engineers means that, for the least well performing engineer (engineer number 4), savings of 32% could be achieved (RE efficiency score of 0.68).

Fig. 1

Engineers’ efficiency scores (Models 1 & 2)

We consider these to be first estimates of the potential savings possible from addressing potential client-side inefficiencies. However, as discussed earlier, the fact that our model is based on bid costs rather than final out-turn costs, limits the extent to which it is possible to make firm recommendations based on the relative efficiency scores. That said, as also discussed earlier, the problem of differences between bid and outturn costs would only be an issue if we see these as non-random, and we noted earlier that evidence for unbalanced bidding is relatively weak in practice.

Results of the nature illustrated by Fig. 1 provide a starting point for a discussion within the organisation of reasons for the differences between the engineers. Some of the gap could turn out to be reasonable, for instance due to that the data analysis lacks information about important external preconditions for the projects. However, the discussion could also address the precise way that some engineers have included activities in the Quote for Bids that others have not. For example, it may be the case that some engineers are specifying larger quantities and better quality than needed given the preconditions. These types of deliberations are central for organisations that seek to be operationally dynamic and learning.

Econometric cost studies also provide insights for how or where to achieve cost savings not only through studying relative efficiency scores. One obvious aspect concerns the possibility of scale economies. Figure 2 plots the elasticity of costs with respect to contract size against contract size (measured in m² of asphalt), calculated from RE estimates. Interpreting the output elasticity in the usual way in the general and transport cost function literature (see e.g. Coelli et al. 2005, Wheat, 2017 and Caves et al. 1985) as an indication of returns scale, the elasticity at the sample mean is 0.69 and the plot shows increasing returns to scale (cost elasticity below unity) for most of the sample “. The straightforward interpretation of this finding then is that it indicates that costs can be reduced by increasing project size. However, it would only make sense to expand the size of a project by re-surfacing adjacent sections if those sections were close to requiring re-surfacing. Therefore in practical terms, increasing the scale of projects would mainly be achieved by grouping multiple objects in an area into one contract. Ridderstedt and Nilsson (2023) examines the cost effects of whether and how this work is grouped by the IM, a case of bundling rather than combinatorial auctions, finding that the extent of economies of scale do not depend on whether the re-surfacing work is carried out one or multiple places (the latter in the same area).

Fig. 2

Estimated elasticities at different amounts of asphalt (RE model)

A potential risk with trying to reduce costs by making contracts larger is that only large contractors can submit bids. The kind of surface renewal technology under scrutiny, hot mix asphalt, requires the bidding firms to have a plant for mixing bitumen and gravel into asphalt. The set-up cost for this plant is substantial and it is not likely that increasing the size of the contracts would further affect this aspect of the market, and therefore this issue should not greatly constrain the potential scale-based cost savings that might arise from larger contracts.

7 Conclusions

This paper has produced new evidence on the relative efficiency performance of road resurfacing projects in Sweden. The paper is new in several respects.

First, the empirical case is novel. While there is a literature on efficiency in the tendering of public sector services, to our knowledge there is no previous literature that compares the performance of road projects at this level of disaggregation. This makes it possible to focus on micro-level efficiency of specific projects. The past literature on roads efficiency has instead focussed on more aggregate measures of cost; this also being generally true in the wider (non-road) efficiency modelling literature.

Second, and importantly, the methodological approach is new, taking the perspective of the tendering agency (client side), and taking as given that the competitive tendering process itself should eliminate other sources of inefficiency. It thus asks whether aspects of the way in which the purchaser specifies the contract causes costs to be unnecessarily high. In taking this perspective the model imposes a non-traditional panel structure, taking the procuring engineer as the unit of analysis from an efficiency perspective, with multiple observations on the performance of the engineers obtained through the multiple projects for which they are responsible (see also Smith et al. 2012 who proposed models to address this type of panel structure).

We regard our results as first estimates of relative efficiency performance at project level, taking the engineer (client-side) perspective. In part they are intended to highlight the client-side, specification aspects of inefficiency, and to demonstrate the methodology. Future development could involve seeking to address the issues caused by utilisation of bid rather than outturn costs for example. These first estimates indicate relatively modest but important efficiency differences between procuring engineers. The least efficient engineer could reduce costs by 32% by matching the performance of the frontier, and on average costs could be reduced by around 20% if all engineers could achieve best practice. This saving would amount to around EUR 40 million per year from Sweden’s annual road resurfacing budget of EUR 200 million.

It is worth noting that the model has a relatively rich specification in terms of included variables (also incorporating regional dummies) to capture unobserved heterogeneity, which should increase confidence in the findings on inefficiency. As noted, the importance of the work is that focus is solely on inefficiency that can be attributed to the way the project is specified by the procurer, given that competitive tendering should drive out remaining inefficiencies. Given the context, our study suggests a valuable additional source of potential efficiency savings. This is of importance for road infrastructure renewal costs beyond Sweden given that previous empirical studies focus on the removal of inefficiency through tendering, rather than on the inefficiency resulting from the client side.

In addition to the analysis of client-side efficiency differences, the results complement recent research (Ridderstedt and Nilsson, 2023) in identifying economies of scale as a promising source for efficiency improvements. Future research could be conducted in this area, to examine, for instance, the impact of tendering larger contracts on the internal procurement costs and the supplier’s market, or comparing different methods for grouping multiple objects into larger contracts. Moreover, with advances in data availability, future research with similar a scope and method could reach further in identifying the causes of the found efficiency differences within the organization.

Appendix

Trans-log regression results

	Dependent variable:
	log(Cost)
	Model 1 (FE)	Model 2 (RE)
	(1)	(2)
log(m²)	1.047 (2.237)	0.146 (2.145)
log(Distance)	0.553 (1.134)	0.181 (1.081)
log(AP150)	−5.780 (6.377)	−3.530 (5.901)
log(ADT)	1.923 (1.505)	1.957 (1.347)
log(IRI)	−1.218 (2.041)	−1.325 (1.870)
log(TrackDepth)	−0.667 (2.430)	−0.802 (2.191)
log(Urban)	−18.903 (14.298)	−18.027 (14.084)
log(Speed)	−18.67 (15.797)	−16.534 (16.945)
log(m²)²	−0.080 (0.050)	−0.053 (0.050)
log(Distance)²	0.001 (0.013)	−0.005 (0.014)
log(AP150)²	0.054 (0.376)	0.029 (0.329)
log(ADT)²	0.057** (0.023)	0.043* (0.022)
log(IRI)²	−0.031 (0.035)	−0.027 (0.030)
log(TrackDepth)²	−0.123 (0.092)	−0.149 (0.087)
log(Urban)²	1.471 (1.829)	1.765 (1.745)
log(Speed)²	2.239 (1.799)	2.054 (1.856)
Engineer 1	0.076 (0.117)
Engineer 2	−0.126 (0.129)
Engineer 3	−0.155 (0.123)
Engineer 4	−0.150 (0.128)
Engineer 5	−0.170 (0.079)
Engineer 6	0.028 (0.263)
Engineer 7	−0.084 (0.055)
Engineer 8	−0.009 (0.091)
Engineer 9	−0.181 (0.173)
Engineer 10	0.103 (0.085)
Engineer 11	−0.177 (0.057)
Engineer 12	0.076 (0.114)
Engineer 13	0.094 (0.130)
Engineer 14	−0.085 (0.100)
Engineer 15	−0.137** (0.037)
Engineer 16	0.087 (0.120)
Engineer 17	0.053 (0.075)
Engineer 18	0.033 (0.093)
Engineer 19	0.251 (0.138)
Engineer 20	0.346** (0.104)
Engineer 21	0.559*** (0.216)
Engineer 22	0.411*** (0.076)
Engineer 23	0.337*** (0.062)
Region I		−0.398 (0.129)
Region II		−0.236 (0.145)
Region III		−0.448 (0.127)
Region IV		−0.519 (0.130)
Region V		−0.233 (0.141)
log(m²) × log(Distance)	0.028 (0.037)	0.045 (0.038)
log(m²) × log(AP150)	−0.072 (0.139)	0.034 (0.119)
log(m²) × log(ADT)	0.058 (0.059)	0.042 (0.056)
log(m²) × log(IRI)	0.049 (0.088)	0.008 (0.068)
log(m²) × log(TrackDepth)	0.112 (0.111)	0.115 (0.098)
log(m²) × log(Urban)	0.214 (0.425)	0.340 (0.416)
log(m²) × log(Speed)	0.189 (0.453)	0.228 (0.430)
log(Distance) × log(AP150)	−0.004 (0.067)	0.034 (0.068)
log(Distance) × log(ADT)	−0.020 (0.031)	−0.016 (0.030)
log(Distance) × log(IRI)	0.070* (0.057)	0.065 (0.050)
log(Distance) × log(TrackDepth)	−0.031 (0.069)	0.029 (0.064)
log(Distance) × log(Urban)	−0.351*** (0.251)	−0.425 (0.238)
log(Distance) × log(Speed)	−0.162* (0.188)	−0.160 (0.175)
log(AP150) × log(ADT)	0.045 (0.191)	0.035 (0.183)
log(AP150) × log(IRI)	0.326 (0.169)	0.275 (0.152)
log(AP150) × log(TrackDepth)	−0.518 (0.292)	−0.351 (0.230)
log(AP150) × log(Urban)	1.088*** (1.473)	0.276 (1.383)
log(AP150) × log(Speed)	1.531*** (1.393)	0.694 (1.259)
log(ADT) × log(IRI)	0.052 (0.067)	0.032 (0.059)
log(ADT) × log(TrackDepth)	−0.129 (0.071)	−0.079 (0.060)
log(ADT) × log(Urban)	−0.275 (0.271)	−0.202 (0.302)
log(ADT) × log(Speed)	−0.748*** (0.324)	−0.672* (0.308)
log(IRI) × log(TrackDepth)	0.089 (0.089)	0.098 (0.094)
log(IRI) × log(Urban)	0.409** (0.398)	0.549 (0.427)
log(IRI) × log(Speed)	−0.165 (0.557)	0.029 (0.491)
log(TrackDepth) × log(Urban)	0.040 (0.512)	−0.003 (0.450)
log(TrackDepth) × log(Speed)	0.431 (0.425)	0.263 (0.361)
log(Urban) × log(Speed)	3.986*** (3.167)	3.646 (3.001)
Constant	44.134 (37.630)	43.785 (40.653)
Engineer dummies	Yes	No
Regional dummies	No	Yes
Observations	233	233
R2	0.911	0.889
Adjusted R2	0.875	0.859
Residual Std. Error	0.234 (df = 165)	0.249 (df = 183)
F Statistic	25.342*** (df = 67; 165)	29.927*** (df = 49; 183)

1. Note: *p < 0.05; **p < 0.01; ***p < 0.001. Engineer clustered standard errors in parenthesis