Acessibilidade / Reportar erro

BINOMIAL REAL OPTIONS MODEL WITH DYNAMIC PROGRAMMING APPLIED TO THE EVALUATION OF RAILWAY INFRASTRUCTURE PROJECTS IN BRAZIL

ABSTRACT

This paper presents a methodology for evaluating public investments in railway infrastructure based on the theory of real options, using the binomial model combined with dynamic programming procedures and Monte Carlo simulation. This modelling was proposed in order to overcome the inefficiencies in the evaluation process of projects conducted by the Brazilian public agencies, as pointed out by World Bank studies. According to these studies, there is evidence that the various managerial and regulatory instances of the Brazilian government have been limited to applying classical analysis techniques, considering only the discounted cash flow indicators, instead of paying more attention to risk issues, as well as to the possibilities of managerial flexibility. The proposed analytical procedures are recommended to support infrastructure investment decisions that can be transferred to the private sector or to guide the formation of public-private partnerships (PPPs).

Keywords:
evaluation of railway infrastructure projects; binomial real options model; dynamic programming

1 INTRODUCTION

Public investments in transport infrastructure projects are intended to provide improvement in social welfare, either by favouring the displacement of individuals, in the case of passenger transportation, or by motivating an increase in the productivity of private capital, in the case of cargo transportation; in this case, increases in private investment and aggregate demand are expected, which will contribute to economic growth and social welfare.

Although improving social welfare is the goal of this type of public investment, doubts remain about its effective scope. The work of Aschauer (1989ASCHAUER DA. 1989. Is Public Expenditure Productive? Journal of Monetary Economics, 23: 177-200.) analyzed the productivity of public expenditures in the United States in the 1970s and 1980s. With regard to public investments in infrastructure, such as roads, highways and airports, the study found empirical evidence that such expenditures were determinants in the productivity of the American national income. In this line of analysis, the study by Rodrigue (2009RODRIGUE JP. 2009. The Geography of Transport Systems. New York: Rutledge.) showed that every dollar invested in the US interstate highway network during the years 1954 to 2001 contributed six dollars to the economy’s productivity.

A study conducted by Rajaram (et al., 2014RAJARAM A, LE TM, KAISER K, KIM JH & FRANK J. (Ed.) 2014. The Power of Public Investment Management: Transforming Resources into Assets for Growth. Washington: The World Bank Group, 184 p.) within developing countries reinforces the importance of public investments in infrastructure in economic growth. However, this study also showed that such investments could provide greater economic development if a series of inefficiencies were corrected, especially from the institutional point of view, such as the following:

  • Often, the project is proposed by a particular ministry and is subject to review by the finance ministry. Weak interaction processes between ministries can lead to delays in project appraisal and implementation;

  • Allocating resources to a project requires a multi-year commitment, which can impose challenges of continuity of purpose in a politically managed budget;

  • Large infrastructure projects often involve acquisition of problem areas and issues of resettlement, environmental protection and complex institutional change, which can result in significant delays and cost increases;

  • Projects can be driven by political considerations and subject to different review criteria, which can reduce the credibility of the ex ante facto assessment process.

Among the inefficiency drivers above mentioned, the last of them stands out and is addressed within the scope of the economic feasibility assessment phase, which guides the investment decision making process.

In the Brazilian case, a study carried out by the World Bank (2009)BANCO MUNDIAL. 2009. Avaliação da Eficiência da Gestão do Investimento Público. Brasília: Banco Mundial. showed technical limitations in the phase of assessment of the economic feasibility of public projects, highlighting deficiencies in the elements:

  1. Strategic guidance and preliminary screening;

  2. Formal evaluation of projects;

  3. Independent review of the evaluation;

  4. Budget and project selection;

  5. Project implementation;

  6. Project adjustments;

  7. Operation of facilities;

  8. Project evaluation.

The World Bank study (2009)BANCO MUNDIAL. 2009. Avaliação da Eficiência da Gestão do Investimento Público. Brasília: Banco Mundial. pointed out that the Brazilian government has achieved improvements in these areas, highlighting the efforts that have been made to qualify staff and in the adoption of analysis techniques that are more appropriate to the reality. However, the study did not make a critical analysis of the methods of economic analysis adopted in the evaluation of projects.

The article by Bock and Trück (2011BOCK K & TRÜCK S. 2011. Assessing Uncertainty and Risk in Public Sector Investment Projects. Technology and Investment, 2: 105-123.) highlighted this limitation in the scope of studies produced by the World Bank and reports a lack of evidence of the application of advanced quantitative models in the evaluation process carried out by public institutions.

In this context, a brief analysis of the content of two technical notes issued by Brazilian transportation regulatory agencies which were related to processes of interaction with the private sector [see ANTT (2009)ANTT - AGÊNCIA NACIONAL DE TRANSPORTE TERRESTRE. 2009. Nota Técnica N° 030/SUREF/2006. Brasília: ANTT. and ANTAQ (2014)ANTAQ - AGÊNCIA NACIONAL DE TRANSPORTE AQUAVIÁRIO. 2014 Nota Técnica N° 7/2014/GRP/SPO/ANTAQ/SEP. Brasília: ANTAQ.] pointed out that, in terms of economic evaluation methods adopted in the analysis of investment projects in transport infrastructure, there was a clear adoption of classical investment analysis, with emphasis on techniques such as Net Present Value (NPV) and Internal Rate of Return (IRR). Finally, in order to gather even more current evidence, an analysis of the evaluation techniques used by the Technical, Economic and Environmental Feasibility Studies (EVTEA) contracted by VALEC Engenharia, Construções e Ferrovias S.A., a company controlled by the Brazilian Federal Government and linked to the Ministry of Infrastructure, was carried out aiming at expanding the Brazilian railway network. Considering the scope of services of OS-20 of contract 019/10 (VALEC, 2018VALEC - ENGENHARIA, CONSTRUÇÕES E FERROVIAS S.A. 2018. Relatório Final - Volumes 1 a 7. Apoio Técnico para Complementação, Adequação, Atualização e Consolidação do Estudo de Viabilidade Técnica, Econômica e Ambiental (EVTEA). Implantação da EF-354, Ferrovia Transcontinental. Brasília, DF: VALEC. Available at: Available at: https://www.valec.gov.br/ferrovias/ferrovia-transcontinental/estudos-de-viabilidade-evtea . Accessed: 20 abr. 21.
https://www.valec.gov.br/ferrovias/ferro...
), it was evidenced that the EVTEA continue applying the same classical evaluation techniques already mentioned, which reinforces that the conclusions of the World Bank study (2009)BANCO MUNDIAL. 2009. Avaliação da Eficiência da Gestão do Investimento Público. Brasília: Banco Mundial. are valid today.

In summary, although the scientific community has improved the techniques of investment analysis, where a more comprehensive view of risk and managerial flexibility could be included in the evaluation by the theory of real options, there is evidence that the Brazilian government, in the most diverse managerial and regulatory instances, has evaluated the economic feasibility of investment projects in transport infrastructure according to the classical viewpoint, focusing on analyses based on techniques of admittedly limited scope, besides attributing little importance to the analysis under conditions of risk and uncertainty. Moreover, it should be noted that these limited procedures have been extended in the analysis of projects in partnership or delegated by public authorities to the private sector (see ANTT, 2009ANTT - AGÊNCIA NACIONAL DE TRANSPORTE TERRESTRE. 2009. Nota Técnica N° 030/SUREF/2006. Brasília: ANTT. and ANTAQ, 2014ANTAQ - AGÊNCIA NACIONAL DE TRANSPORTE AQUAVIÁRIO. 2014 Nota Técnica N° 7/2014/GRP/SPO/ANTAQ/SEP. Brasília: ANTAQ.).

Given these circumstances, this article seeks to answer the following questions regarding the evaluation of public investments in transport infrastructure projects:

  1. How could the approximation between social and private evaluations be achieved, in order to reduce the inefficiency of the public investment decision-making process?

  2. How the application of the real options technique can improve the process of evaluation of public investments in transport infrastructure, so as to bring the areas of public management and regulation closer to the new propositions of the scientific community?

  3. What risk and uncertainty factors should be considered in the assessment as well as in the whole decision-making process?

  4. The inclusion of the notion of managerial flexibility, especially in terms of start-up, postponement, scheduling or abandonment of investments could indeed be implemented and could reduce the inefficiency of the aforementioned managerial process?

Faced with these issues, this paper has the general objective of presenting a methodology for the evaluation of investments in infrastructure for the Brazilian case based on the theory of real options focusing on railway transportation. In this paper, the real options theory is applied using operational research elements, such as dynamic programming and Monte Carlo simulation, as a way to improve the decision making process under analysis.

It is expected that the expected reduction of inefficiencies in the decision-making processes at issue will provide positive effects on economic development and, consequently, on the improvement of Brazilian social welfare.

2 LITERATURE REVIEW

2.1 The contribution of operations research to real options theory and applications

The consideration of real options in the valuation of companies and investment projects is reported in the literature as from the second half of the 1970s. The study by Myers (1977MYERS SC. 1977. Determinants of Corporate Borrowing. Journal of Financial Economics , 5: 147-175.) may be considered one of the forerunners on the subject, having grounded the real options theory (ROT) itself. In his analysis, Myers pointed out that at each point in time the value of a company results from a set of tangible and intangible assets, whereby tangible assets are accumulated units of production capacity, that is, they are real assets, whilst intangible assets are the options to purchase additional units in future periods, characterizing real options as elements of managerial flexibility in uncertain situations

The contributions of operations research to the theory and application of real options have been addressed in the literature since 1979, according to a search conducted in the Scopus database in March 2022 using the words “real options”, considering the period from 1979 to 2022. As a result, 360 published research articles on real options were found in the field of decision sciences, a field that encompasses operations research.

In addition a search of the Web of Science database in the same month of March 2022 with the words “real options” resulted in 948 articles published in operations research and management science between 1991 and 2022, which confirms that operations research has been working on the subject for more than three decades.

The article published by Trigeorgis and Tsekrekos (2018TRIGEORGIS L & TSEKREKOS AE. 2018. Real Options in Operations Research: A Review. European Journal of Operational Research , 270(1): 1-24.) is enlightening as to the contributions of operational research to the theory and practice of real options. In their study, 164 articles published in the five most renowned journals of the field in the world were reviewed, considering: Annals of Operations Research (ANOR), European Journal of Operational Research (EJOR), International Journal of Production Economics (IJPE), Management Science (MS) and Operations Research (OR). The analysis covered the period from 2004 to 2015 and pointed to applications in the fields of: Uncertainty and Investment (19%), R&D, Innovation & Technology (18%), Production and Manufacturing (23%), Supply Chain and Logistics (18%), Energy, Natural Resources and Environment (13%) and Valuation Models and Other Topics (9%).

In this paper, a more recent analysis was conducted, encompassing articles published between 2020 and 2022 in the five journals cited by Trigeorgis and Tsekrekos (2018TRIGEORGIS L & TSEKREKOS AE. 2018. Real Options in Operations Research: A Review. European Journal of Operational Research , 270(1): 1-24.), as a way to attest that the field of operations research continues to contribute to the development of real options theory, which can be seen in the references in Table 1.

Table 1
Recent contributions of operations research to theory and application of real options.

Table 1 presents the most recent articles published on real options in the five journals highlighted above, considering the fields of application and modelling processes adopted. In this cross-section of references, the predominant fields of application were Uncertainty & Investment and Valuation Models & Other, where the application of numerical and analytical methods prevailed, and more than half of the studies involved empirical applications or analyses.

2.2 Evaluation of transport infrastructure projects with real options

Although the contribution of operations research to the theory and applications of real options has been attested in the previous topic, the subject of this paper has been little covered in this field. With the exception of the studies by Silaghi & Sarkar (2021SILAGHI F & SARKAR S. 2021. Agency Problems in Public-Private Partnerships Investment Projects. European Journal of Operational Research , 290(3): 1174-1191.), Jin (et al., 2021JIN H, LIU S, SUN J & LIU C. 2021. Determining Concession Periods and Minimum Revenue Guarantees in Public-Private-Partnership Agreements. European Journal of Operational Research , 291(2): 512-524.) and Thijssen (2022THIJSSEN JJJ. 2022. Optimal Investment and Abandonment Decisions for Projects with Construction Uncertainty. European Journal of Operational Research , 298(1): 368-379.), the recent literature review found no references on the application of real options in transportation infrastructure projects in the field of operations research.

When expanding the literature review to other fields of study it was found that one of the first articles which have applied real options to the case of analysis of investments in transport infrastructure was produced by Kitabatake (2002KITABATAKE Y. 2002. Real Options Analysis of the Minami Alps Forest Road Construction Project: New Valuation Approach to Social Infrastructure Project with Sequential Unit Projects. Environmental Economics and Policy Studies, 5: 261-290.) and focused on the road modal. The paper analyzed project evaluation models in the ex-ante-facto condition considering the existing legal framework in Japan and a new framework that considered the option of abandoning the project totally or partially.

Other studies have been developed involving the application of the real options theory to transport infrastructure projects, covering especially the journals of the engineering and infrastructure fields, as follows: Journal of Construction Engineering and Management (JCEM), Journal of Management in Engineering (JME), Construction Management and Economics (CME), Journal of Infrastructure Systems (JIS), Transport Research Arena (TRA), Transportation Research (TR), Centre for Transport Studies Working Paper (CTSWP), Production (P), International Conference on Applied Economics (ICAE), Economic Annals (EA), Frontiers of Computer Science in China Journal (FCSCJ) and Environmental Economics and Policy Studies (EEPS). Table 2 presents the main information on the articles analysed.

Table 2
Real options theory applied to transport infrastructure projects.

As can be seen in Table 2 about 52% of the 23 articles analyzed focused on Uncertainty & Investment, while the remaining articles were focused on Valuation Models & Other. In terms of modelling, almost all articles operated with numerical methods, with emphasis on Monte Carlo Simulation, which was reported in ten articles, and the Lattice Approach (binomial model), which was addressed in four papers. There were only two mentions of the application of real options in the analysis of railway infrastructure projects, with almost all applications referring to the case of highway projects.

All the studies analysed focused on public-private partnerships (PPPs) projects, with an emphasis on Build-Operate-Transfer (BOT) operations, which are partnerships involving risks that must be properly managed and mitigated. Private partners are especially sensitive to revenue risk, and their attention is focused on the financial viability of the project. Hence, private partners expect the public sector to provide some sort of risk-sharing mechanism in the form of minimum revenue guarantees or abandonment options.

3 METHODS

This section presents the real options theory and the evaluation methodology proposed in this paper.

3.1 Real options theory

The real options theory is a derivation of financial options. Financial options give their buyer, upon payment of a premium C (call) or P (put), the right, but not the obligation, to buy (call) or sell (put) a given financial asset, at a given strike price (X), at a given maturity (T); if the option is European, the exercise of the option takes place only at maturity (T), if the option is American, the exercise of the option can be done also at tT.

The real options theory refers to the fact that the project sponsor has the ability to choose the directions for its investment, whether it is start-up, scaling, postponing operation, or abandonment, throughout the life of the project.

According to Lawrence and Thomas (2008LAWRENCE C & THOMAS M. 2008. Real Options: Applications in Public Economics. MPRA, Paper No. 11915. München: München Universität.), the elements of a financial option can be adapted to real options theory, according to Table 3.

Table 3
Elements of financial options and real options.

Table 3 shows the analogy between real options and financial options. As an example, an investment project can be treated as a call option on the value of the expected cash flows from the investment. Given this analogy, real options are valued using the pricing methods for financial options (see Figure 1), with the necessary adaptations.

Figure 1
Methods for options valuation.

Figure 1 presents a general classification of methods for options valuation. Among the wide range of models that fall under this classification, two of them stand out and bring together the largest number of applications: the Black and Scholes model (1973BLACK F & SCHOLES M. 1973. The Pricing of Options and Corporate Liabilities. Journal of Political Economy, 91: 637-659.) and the binomial model of Cox, Ross and Rubinstein (1979COX JC, ROSS SA & RUBINSTEIN M. 1979. Option pricing: A simplified approach. Journal of Financial Economics, 7(3): 229-263.). While the Black and Scholes model falls into the category of analytical methods of closed solutions, the binomial model is a numerical method of approximation of stochastic processes by the lattice approach.

3.2 Black & Scholes model

The calculation of the premium of a call option (C 0) proposed by Black and Scholes (1973BLACK F & SCHOLES M. 1973. The Pricing of Options and Corporate Liabilities. Journal of Political Economy, 91: 637-659.) considers that the price of the underlying asset (S t ) follows a geometric Brownian motion (GBM) as a stochastic motion pattern. Consider that the continuous rate of return of the underlying asset over a small interval ∆t is defined by r~Δt. Thus, the estimated price for the interval between t e t+Δt is:

S t + Δ t = S t e r ~ Δ t Δ t (1)

Assuming that r~Δt over a short period ∆t is normally distributed with mean µt and variance σ 2t and considering that the term σ 2t can be dimensionally standardized to σΔt, one can estimate the price of underlying asset as

S t + Δ t = S t e μ Δ t + σ Z Δ t (2)

where Z is the random component with standardised normal distribution.

Based on this stochastic process, Black e Scholes (1973BLACK F & SCHOLES M. 1973. The Pricing of Options and Corporate Liabilities. Journal of Political Economy, 91: 637-659.) presented a closed form to calculate de price of a European call option, under the risk-neutral point of view:

C 0 = S 0 N d 1 - X e - r T N d 2 d 1 = ln S 0 X + r + σ 2 2 T σ T d 2 = ln S 0 X + r - σ 2 2 T σ T = d 1 - σ T (3)

where S 0 is the current price of the underlying asset at t 0, N(d) is the probability that the random walk of the standardized normal distribution is less than d, X is the strike price of the option, r is the continuous annual risk-free rate of return, T is the time to maturity of the option (in years) and σ is the volatility of the underlying asset’s returns.

Using an analogy for real options, the option value C 0 in the equation (3) is the net present value (NPV) of the project with embedded real option.

3.3 Binomial model

The binomial model assumes an approximation of the stochastic process represented in equation (2) by an experimental Bernoulli discrete-time process and its evaluation is based on the riskneutral approach. In this experimental process the results are binary: success or failure. Adapting this process to the price movements, we have that the success price movement corresponds to an up in price (u, up) and the failure price movement refers to a down in price (d, down); being that this process is modelled by the binomial probability distribution.

The pricing of an American call in the binomial model is an open process, which is adaptable to the situation in analysis. The pricing process begins with the construction of a portfolio for the issue of a covered call (C), consisting of a fraction of the underlying asset at current price (S) that is financed by borrowing (B) at a risk-free rate (r). Using the binomial lattice, the arbitrage portfolios can be represented as shown in Figure 2.

Figure 2
Arbitrage portfolios for the issuance of a covered call option.

Figure 2 shows that asset prices for portfolios A and B are subject to up (u) and down (d) movements. In the case of asset B (bond), prices in the up and down movements are updated continuously at the same rate, since it is a risk-free asset. For asset C (call), on the other hand, the payoffs are conditioned by the up and down movements. The connection of prices in the up and down movements to the original price is made by the probabilities of up (p) and down 1-p movements and S is calculated recursively:

S = p S u + 1 - p S d e - r Δ t (4)

where e −r∆t corresponds to the continuous discount factor. The solution of equation (4) for p is:

p = e r Δ t - d u - d (5)

According to Hull (2018HULL JC. Options, Futures, and Other Derivatives. 2018. 9 ed. Pearson: Essex, 892 p.), the factors of the up (u) and down (d) movements should incorporate the volatility (σ), thus:

u = e σ Δ t (6)

d = e - σ Δ t = 1 u (7)

Finally, the price of option C is computed recursively from the expected payoffs with the exercise of an American option at tT:

C = ( p ) m a x ( S u - X , 0 ) + 1 - p m a x ( S d - X , 0 ) e - r Δ t (8)

Expanding the analysis to real options and adapting the terminologies of the variables, the recursive process is operated by dynamic programming, according to the equation (see Smith, 2005SMITH JE. 2005. Alternative Approaches for Solving Real-Options Problems. Decision Analysis, 2(2): 89-102.; Brandão et al., 2005BRANDÃO LE, DYER JS & HAHN WJ. 2005. Using Binomial Decision Trees to Solve RealOption Valuation Problems. Decision Analsys, 2(2): 69-88.):

V t , j = C F t , j + p V t + 1 , j + 1 - p V t + 1 , j + 1 e - r Δ t (9)

where V t,j is the present value at the t-th period of the j-th price movement, which is computed by the cash flow received in the same period ( CF t,j ), which is added to the discounted expected value of the next period. Note that this equation does not yet incorporate the real option of the project.

The incorporation of a real option in the recursive equation can be done by considering a hypothetical situation where, in a given period, there is the option to abandon the project receiving the residual value R. In this way, the new equation becomes:

V t , j = m a x C F t , j + p V t + 1 , j + 1 - p V t + 1 , j + 1 e - r Δ t , C F t , j + R (10)

where the present value V t,j will be the maximum value between the project value based on and the cash flow at the t-th period of the j-th price movement plus the residual value of the abandonment option R.

4 THE APPLICATION OF REAL OPTIONS TO A CASE STUDY

4.1 Background to study area

Transport infrastructure covers investments in road, water, railway, air and pipeline modes. Whilst the first four modals deal with cargo and passengers, the last modal only focuses on cargo transportation. In this article, the railway modal was selected for the application of the real options, since the literature review (item 2.2) pointed to a lack of empirical-based studies addressing this type of modal.

In the case of Brazil, the railway network is segregated between the public and private sectors. The public sector railway network is planned and managed by the DNIT (National Department of Transport Infrastructure), an agency linked to the Ministry of Infrastructure (MI) (DNIT, 2021DNIT - DEPARTAMENTO NACIONAL DE INFRAESTRUTURA DE TRANSPORTES. 2021. O Papel do DNIT no Modal Ferroviário. Brasília, DF: DNIT. Available at: <Available at: http://www1.dnit.gov.br/ferrovias/coordenacao.asp > Accessed: 20 abr. 21.
http://www1.dnit.gov.br/ferrovias/coorde...
), while the construction and operation of railway infrastructure is under the responsibility of VALEC Engenharia, Construções e Ferrovias S.A., a privately held company controlled by the Union and also linked to the MI. The case study adopted for the application of real options was hired by VALEC S.A., while technical, economic and environmental feasibility study (EVTEA), and was developed by ENEFER Consultoria, Projetos Ltda., according to the scope of services of OS-20 of contract 019/10 (VALEC, 2018VALEC - ENGENHARIA, CONSTRUÇÕES E FERROVIAS S.A. 2018. Relatório Final - Volumes 1 a 7. Apoio Técnico para Complementação, Adequação, Atualização e Consolidação do Estudo de Viabilidade Técnica, Econômica e Ambiental (EVTEA). Implantação da EF-354, Ferrovia Transcontinental. Brasília, DF: VALEC. Available at: Available at: https://www.valec.gov.br/ferrovias/ferrovia-transcontinental/estudos-de-viabilidade-evtea . Accessed: 20 abr. 21.
https://www.valec.gov.br/ferrovias/ferro...
).

The EVTEA of the EF-354 railroad implementation in the stretch from Mara Rosa/GO to Lucas do Rio Verde/MT is 883 km long and was composed of several studies: engineering studies, environmental insertion studies, market studies, operational studies, socio-economic studies and financial evaluation.

This case study is based on the financial evaluation, which was based on the estimated future cash flows for three prospective scenarios: i) without ferrogrão, which is the base case (the term ferrogrão refers to the transport of cereals in freight trains), ii) with ferrogrão, allocation with flow sharing, and iii) with ferrogrão, allocation all or nothing. Since the two scenarios with ferrogrão are hypothetical, the focus of the analysis in this article concentrates on the baseline scenario (without ferrogrão)

The financial evaluation was performed from the point of view of the future concessionaire winner of the public bid to implement and operate the established railway stretch and the estimated future cash flows are non-leveraged, corresponding to the operational evaluation of the project.

The methodology of financial evaluation was based on traditional analysis, including NPV, IRR, discounted payback period and benefit-cost index (BCI), as established by the company that hired the service (VALEC S.A.). VALEC also defined other parameters of the evaluation, such as the opportunity cost of capital (minimum attractive rate MAR) of 9.57% p.a. (per annum).

The temporal planning horizon considered: i) the year 2018 as t 0, ii) the railway construction between 2019 (t 1) and 2024 (t 6), iii) the entry into operation of the Mara Rosa Água Boa subsection in 2023 (t 5), iv) the coming into operation of the Água Boa Lucas do Rio Verde subsection in 2025 (t 7) and v) the operation term of 30 years as of the operation of the first subsection [from 2023 (t 5) to 2052 (t 34)].

The estimated operational revenue in the scenario without ferrogrão was obtained with the transportation of soy (40.90%), corn (39.59%), diesel oil (3.40%), fertilizer (3.34%), alcohol (2.24%), cotton (1.81%), cement (1.36%) and other products (7.36%). The estimated costs considered maintenance of the permanent railway and systems, maintenance costs for railroad rolling stock, operating costs, operating expenses, depreciation of the investment and amortization of compensation for environmental liabilities. The cash flows of the scenario without ferrogrão, as well as the results of the traditional evaluation performed are shown in Table 4.

Table 4
Projected future cash flows for the scenario without ferrogrão and evaluation indicators (monetary values in R$ 1,000.00).

According to the defined parameters, the analysis based on deterministic cash flows discounted at a risk-adjusted rate (MAR) of 9.57% p.a. pointed to the economic feasibility of the project, with a NPV of R$ 3.571 billion, an IRR of 14.35% p.a., capital recovery in 18 years and BCI of 1.19. The study also developed a sensitivity analysis of cash flows to changes in revenues, costs and investments, considering the ceteris paribus condition, as a risk analysis tool. This analysis identified areas of variation in which the concessionaire could assume the operation as a function of feasibility and areas of unfeasibility, for which the formation of public-private partnerships may be valid.

It is important to highlight that these indicators based on the classic NPV analysis, even if they pointed to the economic feasibility of the project, are considering that, once the investment is made, the concessionaire will have to keep it running until the end of its useful life. However, in the real world, the concessionaire has the possibility of taking managerial decisions that affect the conduct of the project throughout its useful life, including even its abandonment at the cost of contractual penalties. As these possibilities of managerial flexibilities are available in reality, they need to be incorporated into the analysis. This is the purpose of real options analysis.

4.2 Numerical procedures and results

By considering real investment opportunities as collections of options over real assets, the following methodological flow can be established for the proposition of the evaluation model presented in this paper, which is based on the steps suggested by Amram and Kulatilaka (1999AMRAM M & KULATILAKA N. 1999 Real Options. Cambridge-MA: Harvard Business School Press.), as shown in Figure 3 and which are described in the following:

Figure 3
Stages for the application of real options to the evaluation of investment projects.

Stage I: Develop an application framework that describes the possible decisions, quantifying the relative uncertainty and specifying the decision rule.

At this stage, the case study data was used, focusing on the projected net cash flows from the year 2025, as this will be the time when revenues will be full, until the year 2052. With this, a methodology for estimating the uncertainty of net cash flows was developed, according to the following steps:

  1. Estimation of the relationship between cash inflows and outflows. This analysis indicated that outflows represent, on average, 47.46% of project inputs, with a standard deviation of 16.86%, with maximum value of 128.23% and minimum value of 38.13%.

  2. Definition of the functional relationship of the net operating revenue (NOR) of the largest railroad services concessionaire (Rumo S.A.), listed as a benchmark, and influence variables, with the purpose of estimating the future behaviour of the project revenues. As influence variables were defined the price of American crude oil (WTI-USD) and the index of the São Paulo Stock Exchange (BVSP), both for their connection with macroeconomic aggregates. The analysis, based on the multiple regression between the revenues of Rumo S.A. (RAIL3), the oil price (WTI-USD) and the BVSP index pointed to the equation NORRAIL3=831,672.155,882.23WTIUSD+10.72BVSP , whose parameters estimated based on quarterly data in the period between Dec/2010 and Dec/2017 showed statistical significance at 1% and 5%, respectively.

  3. Definition of WTI-USD and BVSP stochastic processes according to the Brownian motion established in equation (2), from their µ and σ and continuous returns ri,t=lnSi,t+Δt/Si,t . This definition, which is based on the hypothesis of market efficiency in the weak form (random walk), was based on the period between Dec/2010 and Dec/2017, on a daily basis with subsequent transformation into annual frequency (p.a.), resulting in the metrics: µ wti-usd=5,12% p.a., σ wti-usd=32,19% p.a., µ bvsp=1,39% p.a., σ bvsp=23,09% p.a..

  4. Calculation of revenues of project as a function of stochastic Brownian motions in WTIUSD and BVSP prices, costs and operating cash flow.

  5. Monte Carlo simulation with 10,000 processes, resulting in 10,000 net cash flows, from which 9,999 continuous returns were extracted. The simulation was run 120 times to evaluate the integrity of the calculated volatility (σ), which stabilized at 24.41% p.a.

Stage II: Implement the option valuation model with the inputs from Step I, such as the current value and volatility of the underlying asset.

This stage can be broken down into the following steps, according to Copeland and Antikarov (2003COPELAND T & ANTIKAROV V. 2003. Real Options: a Practitioner’s Guide. New York: Texere.):

  • a) Calculate the NPV of the project without the flexibility real option. The application begins with calculations of the binomial model parameters and the results of which are shown in Table 5.

  • The calculation of the NPV ( NPV=t=0TCFte-tr, where CF t is the t-th project cash flow, e is the continuous discount factor, t-th is the discount period) without managerial flexibility at the risk-free interest rate (r) returns a value of R$7.359 billion, indicating the economic feasibility of the project in a risk-neutral context.

  • In applying the binomial model, the net cash flows from period 7 (as a function of the start of full operation) to period 34 are considered, and the NPV calculation results in R$ 12.238 billion (see the second row of table 5), which is the starting point of the binomial lattice (see table 6).

  • As the value of the project will be based on its cash flows, we adopted the calculation of the payout rate (δ) proposed by Brandão (et al., 2005BRANDÃO LE, DYER JS & HAHN WJ. 2005. Using Binomial Decision Trees to Solve RealOption Valuation Problems. Decision Analsys, 2(2): 69-88.), δt,j=CFt,j/Vt,j , where V t,j is the value of the project in the t-th period, considering the present value of future cash flows, and CF t,j is the cash flow in the t-th period. This rate is intended to calculate the cash flows that are paid out at the end of each period as a function of the project value.

Table 5
Parameters for the real options analysis (monetary values in R$ 1,000.00).
Table 6
Cut-off of payout rates (δ t,j ) and of the binomial lattice of project values (V t,j ) from years 7 to 20 without real options (in R$ 1,000.00) - the value in year 6 is discounted at t 0.
  • b) Build an event tree to model the uncertainties. It was chosen to follow the recommendations of Smith (2005SMITH JE. 2005. Alternative Approaches for Solving Real-Options Problems. Decision Analysis, 2(2): 89-102.) and Hull (2018HULL JC. Options, Futures, and Other Derivatives. 2018. 9 ed. Pearson: Essex, 892 p.), which indicate that the binary lattice is of more accessible use, being of easy implementation in spreadsheets. This is a favourable point, since all calculations performed in this paper were implemented in Windows/Excel, demonstrating the non-necessity of using specific software.

  • Table 6 contains the calculated payout rates (δ t, j ) and a cut-off of the binomial lattice constructed for periods 7 to 20, containing the values without real options. The calculation procedures were based on the work of Brandão (et al., 2005BRANDÃO LE, DYER JS & HAHN WJ. 2005. Using Binomial Decision Trees to Solve RealOption Valuation Problems. Decision Analsys, 2(2): 69-88.). The illustrative cut-off is necessary since the lattice corresponds to a 29×29 matrix.

  • The binary lattice presented in Table 6 is a preliminary representation showing the definition of project value in a scenario where volatility is embedded in price movements that unfold as exposure to the future increases over the planning time horizon. The calculations begin with the present value of the net cash flows in the interval from year 7 to year 34 and are expanded to the right based on the relationships Vt,ju=Vt-1,j-Vt-1,jδt-1,ju and Vt,jd=Vt-1,j-Vt-1,jδt-1,jd , which results in the binary lattice of project values in the various up and down price movements (V i ).

  • Taking into account the principles of the binomial model, the binary lattice of cash flows is calculated, considering that CFt-1,j=Vt-1,jδt-1,j , as can be seen in Table 7.

Table 7
Cut-off of the binomial lattice of the project cash flows (F t,j ) from years 7 to 20 (in R$ 1,000.00).
  • c) Build a decision tree to identify and integrate possible management flexibilities. In order to illustrate the inclusion of real management options in the analysis, under the conditions of an American option, consider hypothetically that the concessionaire may abandon the project in years 10, 15, 20, or 25, transferring it to the Federal Government or a private entity, receiving a cash flow R corresponding to the residual value net of contractual penalties, in amounts related to the investments operated between years 1 and 6, monetarily restated until the abandonment date, as shown in table 8, where the rates employed (row 3) are mere assumptions. The present value in t 0 of the infrastructure and operating investments made between years 1 and 6 is R$ 5.269 billion.

Table 8
Hypothetical values and conditions of cash flows (R) of the abandonment periods (in R$ 1,000.00).
  • d) Calculate the new project value with and without the management flexibility options, which are the total project value and the real option value. Based on the hypothetical abandonment values and conditions in years 10, 15, 20, or 25, we use the recursive equation

  • e) in each of the forecasted years, generating four binomial lattices calculated by dynamic programming. To illustrate the process, Table 9 shows the binomial lattice with the abandonment option for year 10.

  • The values of the real options under each of the abandonment scenarios (equation (10)) are shown in Table 10. The table also shows the impacts of the real options on the project NPV under each of the abandonment scenarios. In all the projected scenarios, the project economic feasibility was improved when the abandonment option was considered (see row 5), indicating that managerial flexibility brings economic benefits to the project.

Table 9
Cut-off of binomial lattice of project values (V t,j ) from years 7 to 20 with the real option of abandonment at time t 10 (in R$1,000.00) - the value in year 6 is discounted at t 0.
Table 10
Values of the option to abandon and impacts on the project NPV (in R$ 1,000.00).

The other stages III and IV presented on Figure 3 were not applied in this paper, as they refer to real decision-making processes.

4.3 Conclusive analysis

The real options approach is broad enough to address more complex situations than the one illustrated in this paper, considering multiple time periods, where cash flows may be associated with commodity prices, for example, and uncertainties include future costs, discount rate, and the structural and parametric behaviour of cash flows.

In general terms, real options take into account the uncertainty inherent to the decision-making process in a more appropriate manner, as well as incorporating the capacity of managers to act in the project operation after its implementation, in an ex-ante-facto situation, as opposed to the classic analysis, which only considers the situation in which the project is put into operation and will continue to operate until the end of its useful life, regardless of changes in market conditions.

5 FINAL CONSIDERATIONS

5.1 Implications of the presented methodology for infrastructure investment decisions

This paper presented an analytical framework and numerical procedures for the evaluation of public investments in transport infrastructure based on the theory of real options, which was implemented with contributions from elements of operations research, as a way to improve the efficiency of this kind of decision-making process, incorporating a methodology of recognized scientific value in risk analysis.

The analytical framework and numerical procedures presented in Section 4.2 represent the contributions of this paper to operations research, with respect to its inclusion in studies on the theory and application of real options. The numerical techniques proposed present contributions from operations research in the area of Uncertainty & Investment, particularly in terms of the procedures used to define the parameters for the Monte Carlo simulation, which generated volatility estimates of the project’s future cash flows.

There were also contributions to operational research in Valuation Models & Other through the implementation of specific numerical procedures of the lattice approach (binomial model) proposed by Brandão (et al., 2005BRANDÃO LE, DYER JS & HAHN WJ. 2005. Using Binomial Decision Trees to Solve RealOption Valuation Problems. Decision Analsys, 2(2): 69-88.), Smith (2005SMITH JE. 2005. Alternative Approaches for Solving Real-Options Problems. Decision Analysis, 2(2): 89-102.) and Hull (2018HULL JC. Options, Futures, and Other Derivatives. 2018. 9 ed. Pearson: Essex, 892 p.), which culminated with the evaluation of the abandonment option in different prospective scenarios, considering as focus of application a transport modal little addressed in the literature.

The analytical framework presented is recommended for infrastructure investment decisions that can be transferred to the private sector or to guide the formation of public-private partnerships, when the projects are not sufficiently feasible to potential concessionaires.

The inclusion of managerial flexibility in the ex-ante-facto analysis can bring greater efficiency to the decision-making process, since besides real options for start-up, staggering, postponement of operation or abandonment, alternative courses of action can be considered according to, for instance, demands from groups influenced by the project or by new environmental performance requirements.

In the case analysed, the real options for abandonment in the projected periods may favour the inclusion of contractual clauses and conditions for possible abandonment by the concessionaire, as well as procedures for calculating the value of contractual penalties.

The analysis methodology presented, as a risk analysis tool, is superior to the use of other tools of limited reach, such as sensitivity analysis. Sensitivity analysis starts from the assumption of uncertainty, which foresees the full ignorance of the historical behaviour of the variables involved in the decision making process. On the other hand, the risk analysis developed in this paper showed that it is feasible to use proxies that enable the conversion of a scenario of uncertainty into a situation that allows a greater approximation with the changes that occur in the real world, especially in variables that influence the project.

The final results of the modelling process evidenced that the inclusion of real options in the analysis may favour project evaluation, since they increased project feasibility in the four hypothetical situations considered, which attests to possible contributions to increasing the efficiency of this type of decision-making process.

As mentioned at the start of this paper, it is hoped that the expected reduction of inefficiencies in the decision making processes in question will result in positive impacts on economic development, especially in countries with acknowledged weaknesses in their analysis methodologies, as is the case of Brazil.

Finally, it is important to highlight that no criticism is directed to the company that developed the financial evaluation of the case study in this article, which performed the analysis according to the methods and parameters defined by VALEC, which only represented the procedures adopted by the Brazilian public sector.

5.2 Recommendations

It is necessary that public agencies that operate only the analysis of projects in the classical way improve their financial evaluation procedures by including the analysis of real options, generating demand for the definition of new elements of analysis to be included in the notices and bids for public service concessions. Such procedures may favour the future concessionaires themselves, by presenting a broader view of the risks associated with the investment and the possibility of contemplating in the analysis managerial actions to curb potential losses.

Although real options are a research topic developed by academia over 35 years ago, its models are still not known by many senior managers, both in the public and private sectors, who have been more likely to use the traditional NPV approach (Schulmerich, 2010SCHULMERICH M. 2010. Real Options Valuation: The Importance of Interest Rate Modelling in Theory and Practice. 2. ed. Springer-Verlag: Heidelberg, 407 p.), which justifies the continuity of studies on the subject.

Acknowledgements

The author wishes to acknowledge the financial support provided by the University of Brasília (Edital DPI/DPG nº 003/2020).

References

  • ALEXANDER C & CHEN X. 2021. Model Risk in Real Option Valuation. Annals of Operations Research, 299: 1025-1056.
  • ALIBEIKI H & LOTFALIEI B. 2022. To Expand and to Abandon: Real Options Under Asset Variance Risk Premium. European Journal of Operational Research, 300(2): 771-787.
  • ALMASSI A, MCCABE B & THOMPSON M. 2013. Real Options-Based Approach for Valuation of Government Guarantees in Public-Private Partnerships. Journal of Infrastructure Systems, 19(2): 196-204.
  • AMRAM M & KULATILAKA N. 1999 Real Options. Cambridge-MA: Harvard Business School Press.
  • ANTAQ - AGÊNCIA NACIONAL DE TRANSPORTE AQUAVIÁRIO. 2014 Nota Técnica N° 7/2014/GRP/SPO/ANTAQ/SEP. Brasília: ANTAQ.
  • ANTT - AGÊNCIA NACIONAL DE TRANSPORTE TERRESTRE. 2009. Nota Técnica N° 030/SUREF/2006. Brasília: ANTT.
  • ASCHAUER DA. 1989. Is Public Expenditure Productive? Journal of Monetary Economics, 23: 177-200.
  • BANCO MUNDIAL. 2009. Avaliação da Eficiência da Gestão do Investimento Público. Brasília: Banco Mundial.
  • BLACK F & SCHOLES M. 1973. The Pricing of Options and Corporate Liabilities. Journal of Political Economy, 91: 637-659.
  • BLANK FF, SAMANEZ CP, BAIDYA TKN & DIAS MAG. 2016. Economic Valuation of a Tool Road Concession with Traffic Guarantees and the Abandonment Option. Production, 26(1): 3953.
  • BOCK K & TRÜCK S. 2011. Assessing Uncertainty and Risk in Public Sector Investment Projects. Technology and Investment, 2: 105-123.
  • BRANDÃO LE, DYER JS & HAHN WJ. 2005. Using Binomial Decision Trees to Solve RealOption Valuation Problems. Decision Analsys, 2(2): 69-88.
  • CHEAH CYJ & LIU J. 2006. Valuing Governmental Support in Infrastructure Projects as Real Options Using Monte Carlo Simulation. Construction Management and Economics, 24: 545-554.
  • CHIARA N, GARVIN MJ & VECER J. 2007. Valuing Simple Multiple-Exercise Real Options in Infrastructure Projects. Journal of Infrastructure Systems , 13(2): 97-104.
  • CHIARA N & GARVIN MJ. 2008. Variance Models for Project Financial Risk Analysis with Applications to Greenfield BOT Highway Projects. Construction Management and Economics , 26: 925-939.
  • CONG LW. 2020. Timing of Auctions of Real Options. Management Science, 66(9): 1-21.
  • COPELAND T & ANTIKAROV V. 2003. Real Options: a Practitioner’s Guide. New York: Texere.
  • COX JC, ROSS SA & RUBINSTEIN M. 1979. Option pricing: A simplified approach. Journal of Financial Economics, 7(3): 229-263.
  • CUCCHIELLA F, D’ADAMO I & GASTALDI M. 2008. Highway Project Management Through Real Option. International Conference on Applied Economics: 217-223.
  • CUI Q, JOHNSON P, QUICK A & HASTAK M. 2008. Valuing the Warranty Ceiling Clause on New Mexic Highway 44 Using a Binomial Lattice Model. Journal of Construction Engineering and Management, 134(1): 10-17.
  • DEENEY P, CUMMINS B, HEINTZ K & PRYCE MT. 2021. A Real Options Based Decision Support Tool for R&D Investment: Application to CO2 Recycling Technology. European Journal of Operational Research , 289(2): 696-711.
  • DNIT - DEPARTAMENTO NACIONAL DE INFRAESTRUTURA DE TRANSPORTES. 2021. O Papel do DNIT no Modal Ferroviário. Brasília, DF: DNIT. Available at: <Available at: http://www1.dnit.gov.br/ferrovias/coordenacao.asp > Accessed: 20 abr. 21.
    » http://www1.dnit.gov.br/ferrovias/coordenacao.asp
  • DOAN P & MENYAH K. 2013. Impact of Irreversibility and Uncertainty on the Timing of Infrastructure Projects. Journal of Construction Engineering and Management , 139(3): 331-338.
  • FITCH GJ, ODEH I, KAUTZ F & IBBS CW. 2018. System Dynamics Models for the Valuation of Real Options in P3 Concession Project Agreements. Journal of Infrastructure Systems , 24(1): 1-13.
  • GALERA AL, SOLIÑO AS & ABAD BG. 2018. Sharing Risks in Toll Motorway Concessions: Subsidies as Real Options on Traffic Volume. Journal of Infrastructure Systems , 24(4): 1-8.
  • GAO Y & DRIOUCHI T. 2013. Incorporating Knightian Uncertainty into Real Options Analysis: Using Multiple-Priors in the Case of Rail Transit Investment. Transportation Research Part B, 55: 23-40.
  • HOMMEL U & LEHMANN H. 2001. Die Bewertung von Investitionsprojekten mit dem Realoptionsansatz - Ein Methodenüberblick. In Hommel, M., Scholich, Vollrath, R. (ed) Realoptionen in der Unternehmenspraxis: Wert schaffen durch Flexibilität. Springer-Verlag, Berlin: 113-119.
  • HUANG YL & PI CC. 2013. Real-Option Valuation of Build-Operate-Transfer Infrastructure Projects under Performance Bonding. Journal of Construction Engineering and Management , 140(5): 1-7.
  • HULL JC. Options, Futures, and Other Derivatives. 2018. 9 ed. Pearson: Essex, 892 p.
  • JIN H, LIU S, SUN J & LIU C. 2021. Determining Concession Periods and Minimum Revenue Guarantees in Public-Private-Partnership Agreements. European Journal of Operational Research , 291(2): 512-524.
  • KITABATAKE Y. 2002. Real Options Analysis of the Minami Alps Forest Road Construction Project: New Valuation Approach to Social Infrastructure Project with Sequential Unit Projects. Environmental Economics and Policy Studies, 5: 261-290.
  • KRÜGER NA. 2012. The Impact of Stochastic Properties of Traffic Demand on Real Option Value in Road Projects. Centre for Transport Studies Working Paper, 17: 1-32.
  • LAWRENCE C & THOMAS M. 2008. Real Options: Applications in Public Economics. MPRA, Paper No. 11915. München: München Universität.
  • LV J, YE G, LIU W, SHEN L & WANG H. 2015. Alternative Model for Determining the Optimal Concession Period in Managing BOT Transportation Projects. Journal of Management in Engineering, 31(4): 1-7.
  • MAC CAWLEY A, CUBILLOS M & PASCUAL R. 2020. A Real Options Approach for Joint Overhaul and Replacement Strategies with Mean Reverting Prices. Annals of Operations Research , 286: 303-324.
  • MAIER S. 2021. Re-evaluating Natural Resource Investments under Uncertainty: An Alternative to Limited Traditional Approaches. Annals of Operations Research , 299: 907-937.
  • MYERS SC. 1977. Determinants of Corporate Borrowing. Journal of Financial Economics , 5: 147-175.
  • NOORIZADEH A, KUOSMANEN T & PELTOKORPI A. 2021. Effective Purchasing Reallocation to Suppliers: Insights from Productivity Dynamics and Real Options Theory. International Journal of Production Economics, 233: 1-14.
  • PARK T, KIM B & KIM H. 2013. Real Option Approach to Sharing Privatization Risk in Underground Infrastructures. Journal of Construction Engineering and Management , 139(6): 685-693.
  • RAJARAM A, LE TM, KAISER K, KIM JH & FRANK J. (Ed.) 2014. The Power of Public Investment Management: Transforming Resources into Assets for Growth. Washington: The World Bank Group, 184 p.
  • RAKIC B & RADENOVIC T. 2014. Real Options Methodology in Public-Private Partnership Projects Valuation. Economic Annals, LIX(200): 91-113.
  • RODRIGUE JP. 2009. The Geography of Transport Systems. New York: Rutledge.
  • SCHRÖDER D. 2020. Real Options, Ambiguity, and Dynamic Consistency - A Technical Note. International Journal of Production Economics, 229: 1-3.
  • SCHULMERICH M. 2010. Real Options Valuation: The Importance of Interest Rate Modelling in Theory and Practice. 2. ed. Springer-Verlag: Heidelberg, 407 p.
  • SILAGHI F & SARKAR S. 2021. Agency Problems in Public-Private Partnerships Investment Projects. European Journal of Operational Research , 290(3): 1174-1191.
  • SMITH JE. 2005. Alternative Approaches for Solving Real-Options Problems. Decision Analysis, 2(2): 89-102.
  • THIJSSEN JJJ. 2022. Optimal Investment and Abandonment Decisions for Projects with Construction Uncertainty. European Journal of Operational Research , 298(1): 368-379.
  • TORRES-RINCÓN S, VILLARRAGA DF & SÁNCHEZ-SILVA M. 2020. Conceptual and Numerical Analysis of Flexibility in Infrastructure Systems. Journal of Infrastructure Systems , 26(2): 1-18.
  • TRIGEORGIS L & TSEKREKOS AE. 2018. Real Options in Operations Research: A Review. European Journal of Operational Research , 270(1): 1-24.
  • VALEC - ENGENHARIA, CONSTRUÇÕES E FERROVIAS S.A. 2021. Competências. Brasília, DF: VALEC. Available at: Available at: https://www.valec.gov.br/a-valec/institucional/competencias Accessed: 20 abr. 21
    » https://www.valec.gov.br/a-valec/institucional/competencias
  • VALEC - ENGENHARIA, CONSTRUÇÕES E FERROVIAS S.A. 2018. Relatório Final - Volumes 1 a 7. Apoio Técnico para Complementação, Adequação, Atualização e Consolidação do Estudo de Viabilidade Técnica, Econômica e Ambiental (EVTEA). Implantação da EF-354, Ferrovia Transcontinental. Brasília, DF: VALEC. Available at: Available at: https://www.valec.gov.br/ferrovias/ferrovia-transcontinental/estudos-de-viabilidade-evtea Accessed: 20 abr. 21.
    » https://www.valec.gov.br/ferrovias/ferrovia-transcontinental/estudos-de-viabilidade-evtea
  • VASUDEVAN V, PRAKASH P & SAHU B. 2018. Options Framework and Valuation of Highway Infrastructure under Real and Financial Uncertainties. Journal of Infrastructure Systems, 24(3): 1-9.
  • ZHANG G, GUO J, CHAI G, GONG L & WANG X. 2010. Frontiers of Computer Science in China Journal, 4(2): 263-270.

Publication Dates

  • Publication in this collection
    13 June 2022
  • Date of issue
    2022

History

  • Received
    27 Jan 2022
  • Accepted
    14 Mar 2022
Sociedade Brasileira de Pesquisa Operacional Rua Mayrink Veiga, 32 - sala 601 - Centro, 20090-050 Rio de Janeiro RJ - Brasil, Tel.: +55 21 2263-0499, Fax: +55 21 2263-0501 - Rio de Janeiro - RJ - Brazil
E-mail: sobrapo@sobrapo.org.br