EXPEDITING THE TRANSITION FROM NON-RENEWABLE TO RENEWABLE ENERGY VIA OPTIMAL CONTROL

. Much recent climate research suggests that the transition from non-renewable to renewable energy should be expedited. To address this issue we use an optimal control model, based on an integrated assessment model of climate change that includes two types energy production. After setting up the model, we derive necessary optimality conditions in the form of a Pontrya-gin type Maximum Principle. We use a numerical discretization method for optimal control problems to explore various policy scenarios. The algorithm allows to compute both state and co-state variables by providing a consistent numerical approximation for the adjoint variables of the various scenarios. Our numerical method applies to control and state-constrained control problems as well as to delayed control problems. In the policy scenarios, we explore ways how the transition from non-renewable to renewable energy can be expedited.


1.
Introduction. Research of climate scientists has shown that CO 2 emissions are the major cause of a rising greenhouse gas concentration in the atmosphere leading to a changed climate on the Earth. Empirical research has demonstrated that up to 80 percent of CO 2 emissions comes from fossil energy related activities such as the extraction and use of fossil energy for generating electricity, heating, cooling, and transport. Empirical research demonstrates that in particular in manufacturing production the use of fossil energy is highly correlated with CO 2 emissions, see Mittnik et al. [21].
Since the use of fossil energy creates extensive externalities and contributes to climate change, green energy with no such externalities is viewed as backstop technology. Hence, substituting polluting fossil energy by non-polluting energy could be of great help in alleviating the problem of global warming. Major substitutes for fossil energy are renewable energy sources such as solar, wind, water, geothermal and biomass energy. Developing a more extensive renewable energy sector that substantially reduces CO 2 emissions seems to be the most promising route to reduce global warming. It allows economies to substitute away from non-renewable fossil energy, such as coal and crude oil, the prices of which appear to be rising in the long-run, see Greiner, Semmler and Mette [10]. Thus, can such a transition take place before fossil energy is completely exhausted?
The literature on modeling the exhaustion of resources starts with Hotelling [15], where there is no amenity value of resources considered in preferences. Amenity value of resources in preferences are considered in Krautkraemer [16]. Here then the question is asked under what condition it is optimal to preserve some natural resources if there is a positive effect of those on the utility of households. Capital endowment can be an important determinant of resources preservation if there is a capital-resource substitution. The issue of negative externalities of the use of resources, such as the effect of CO 2 emission on the atmosphere, starts to be discussed in growth models in Hoel and Kverndokk [14]. They also built into their model a renewable energy resource, as backstop technology, and admit a law of motion for the atmospheric stock of carbon. They show that, in the presence of greenhouse effects in preferences in an optimal growth model, this may slow down the extraction of fossil fuel -leaving more natural resources in situ.
Subsequent literature has studied further such transition dynamics. Van der Ploeg and Withagen [24], for example, study a model where energy may be produced using a polluting non-renewable resource or a non-polluting renewable energy source that requires resources but not a capital stock. They investigate when it is optimal to switch to the renewable energy production and whether renewable and nonrenewable energy are used simultaneously. They are also interested in the question of whether it is optimal to leave deposits of the non-renewable resource in situ. In a different approach, van der Ploeg and Withagen [23] analyze the optimal use of two non-renewable resources, one implying an extensive emission of greenhouse gases, such as coal, and one source with less, such as oil. It is demonstrated that the sup-optimal market solution uses too much of the more polluting energy source unless the government corrects this market failure.
Our subsequent model is built on the above literature by exploring policies that can expedite the transition from non-renewable to renewable energy. We extend an integrated assessment model of the Nordhaus [22] type by including two types of energy sources as the factors of production for final output. We assume that energy can be produced either from a non-renewable energy source such as fossil fuel or from a renewable source that requires investment in a capital stock. While the renewable energy does not emit greenhouse gases and, thus, does not contribute to global warming, burning fossil fuel entails high CO 2 emission and results in damages to welfare. Those negative externalities from fossil energy consumption can be reduced when renewable energy is phased in sufficiently early.
In our model damages multiplicatively interact with consumption in the household's welfare function. This is different from Nordhaus [22], where damages affect production, and also from Greiner, Grüne, Semmler [9] where, in the numerical explorations, the damage term enters only linearly in the preferences. Given this more complex setting, we focus on policies that may help to expedite the transition to renewable energy. Essential in this context are the extraction cost of the non-renewable resource, the efficiency of the non-renewable energy resource, relative to the renewable resource, the initial endowments of capital and nonrenewable resource, and the damaging effects that non-renewable energy, such as fossil fuel, can create. Of further importance are the constraints on state and control variables, their delayed impact on the system dynamics, the discount rate 1 , and the time horizon of the optimization process.
As to the numerical solution method we do not use here, as Greiner et al. [9] Nonlinear Model Predictive Control (NMPC) as solution algorithm, but rather another method that proves to be very flexible to answer the policy issues above raised. As in Greiner et al. [9] the model is analyzed for infinite time but the numerics is undertaken for a finite time decision horizon. We employ here the Applied Modeling Programming Language AMPL [5] and the optimization solver IPOPT [27] that allow for high speed computations of finite time solutions. AMPL/IPOPT permit easily to explore model variants with highly non-linear feedback structures, a larger number of control and state variables, state and control constraints and delays.
The remainder of the paper is organized as follows. Section 2 summarizes the structure of the model introduced by Greiner et al. [9] and explores the properties of the socially optimally solutions for the multiplicative welfare function. Section 3 introduces the solution method of the different model variants. Section 4 presents the results from our numerical procedure. Section 5 concludes the paper.
2. The canonical control model and its optimal solution. The basic model presented is the same as in Greiner et al. [9], but we here focus on the multiplicative interaction of damages and consumption.
2.1. The basic model. The total flow of energy output E arises as the sum of energy produced from a non-polluting energy sector, creating E n , and from a polluting energy sector, producing the flow of energy, E p . The underlying production functions for the production of the two types of energy are: with u(t) the amount of fossil fuels used at time t to generate energy and K a stock of capital that produces energy using renewable sources of energy such as wind or solar energy and A i , i = p, n, denote efficiency indices. Total energy E consists of the sum of these two types of energy. Note that production of the final good Y (t) uses energy and is a concave function of energy input. In the following we delete the time argument t as long as no ambiguity arises.
with 0 < α ≤ 1, A > 0. Note that energy is a homogeneous good so that modeling the two types as perfect substitutes can be justified. The stock of non-renewable energy source evolves over time according to the following law of motion: As far as the accumulation of total capital is concerned, with a constant decay rate of δ, we have the following: with C consumption and a > 0 gives cost of extracting one unit of the non-renewable resource. The cost function will be modified later.
The use of the non-renewable resource leads to an increase of greenhouse gases (GHGs), M , above its pre-industrial level M o . The greenhouse gas concentration evolves according tȯ where µ ∈ (0, 1) is the inverse of the atmospheric lifetime of greenhouse gases and β 1 ∈ (0, 1) gives that part of greenhouse gases that is not taken up by oceans. The parameter κ > 1 captures the fact that greenhouse gas stabilization is possible only at values exceeding the pre-industrial level. The goal is to achieve stabilization at a doubling of GHGs which would imply κ = 2 in our setting. The control u represents the part of emissions that can be controlled by some planner. But even for u = 0 the GHG concentration can rise on the transition path, i.e. for M < κM o , since other sources emit CO 2 that are beyond the influence of the planner.
As to the utility function U we use a generalization of the one presented in Byrne [3] and adopt the following function that is also resorted to in Smulders and Gradus [26] and Greiner et al. [9], for example: The parameter 1/σ > 0 denotes the inter-temporal elasticity of substitution of consumption between two points in time and ξ > 0 gives the (dis)utility of the greenhouse gas concentration exceeding the pre-industrial level. In Greiner et al. [9] the solutions are discussed and numerically solved for σ = 1 where the utility function is logarithmic in consumption and damages. The here explored version is the one with the nonlinear feedback structure in the preferences. Note that in the above preferences the damages will rise and the marginal utility will fall with ξ rising if the inter-temporal elasticity of substitution is larger than one. For details see Greiner et al. [9]. Moreover, we here solely focus on the solution of the social planner's problem.
2.2. The optimal control model. We can look for the solution of the allocation problem faced by a benevolent social planner taking into consideration the accumulation of greenhouse gases, equation (6). Thus, for a fixed finite horizon T > 0, we have for the planning version the following optimal control problem, where the state variable is denoted by X = (K, R, M ) ∈ IR 3 : Find piecewise continuous control functions C : [0, T ] → IR + and u : [0, T ] → IR + that maximize the cost functional subject to the dynamic equations (9), the control constraints and terminal constraints of the form In (b), R f > 0 denotes a prescribed positive lower bound of the resource. Through the discussion of necessary optimality condition in the next section it will become clear, why a positive lower bound C min for the consumption C(t) is meaningful. Moreover, we shall impose the additional state constraint for the CO 2 concentration for some selected upper bounds M max . This state constraint has order one, since the control u appears in the first derivative of M ; cf. the definition of the order of a state constraint in [11,19].
As we shall see below, the preceding control does not possess a stationary state X * = (K * , R * , M * ) in all components. We can only determine stationary values K * and M * . Hence, there is no clear definition of an infinite-horizon optimal control problem, where we seek to maximize subject to the dynamics (9) and control constraints (10). Admissible infinite-horizon controls will be briefly discussed in Section 3.3.

Necessary conditions of Pontryagin's Maximum Principle.
3.1. Finite-horizon optimal control problem. We evaluate the necessary optimality conditions of Pontryagin's Maximum Principle [13,25] for the finite-horizon control problem (8)- (12). Part of this analysis can be found in Greiner et al. [9]. However, due the boundary conditions (11), the control constraints (10) and the state constraint (12) we need a more refined analysis. Let us first consider the control problem without the state constraint (12). The current-value Hamiltonian function is written as where λ = (λ 1 , λ 2 , λ 3 ) is the adjoint variable, i.e, λ i , i = 1, 2, 3, are the shadow prices of capital, fossil energy and GHG concentration, respectively. The adjoint differential equations arė The transversality conditions for λ(T ) ∈ IR 3 depend on the cases (a), (b), (c) in the terminal constraints (11) and are given by

HELMUT MAURER AND WILLI SEMMLER
To compute the controls C and u that maximize the Hamiltonian (14) subject to the control constraints (10), we compute the so-called free controls C f ree (X, λ) and u f ree (X, λ) which are determined by the equations This yields the free controls Remark. We will choose the pre-industrial level of CO 2 emission as initial condition M (0) = M o = 1, since we are interested in the deviation from this level during the control process which is also incorporated in the cost function (8). However, then the consumption C can not be determined from the equation ∂H/∂C = 0 at t = 0. For that reason, we have introduced the lower bound C min > 0 in (10).
In view of the control constraints (10), the maximizing controls are given by Equation (16) shows that the shadow price of the non-renewable energy source exponentially rises at the rate ρ, i.e. λ R (t) = λ R (0) · e ρ t , giving the Hotelling rule. The shadow price of capital, λ K , is positive while that of GHGs, λ M , is negative since GHGs above the pre-industrial level, to which we limit our considerations, lead to welfare losses. The extraction rate u will be positive only if the marginal product of u in energy production exceeds its cost which consist of the unit extraction cost, a, plus its price relative to the shadow price of capital, (λ R − λ M β 1 )/λ K . It should be noted that the price of the resource consists of the shadow price of the resource, λ R , plus the effective price of GHGs, −λ M β 1 , where effective means that the shadow price of GHGs must be multiplied by, β 1 , since only 0 < β 1 < 1 of GHG emissions enter the atmosphere while the rest is absorbed by oceans.
On the other hand, if the cost of investment in the renewable energy capital stock is large there may be a time period when investment equals zero and only the polluting resource is used to generate energy. This will occur when the cost of resource extraction is low relative to investment in renewable energy which can be the case for low extraction costs and for small marginal damages of GHGs while productivity of the non-polluting energy capital is low.
For C > C min , the growth rate of consumption can be derived from (19) and (21) asĊ That equation demonstrates that on the transition path the growth rate of consumption is higher if consumption and damages from global warming are substitutes compared to the case when consumption and damages are complements, for a given growth rate of the shadow price of capital. This holds because the planner will put a higher weight on raising consumption when GHGs increase since the negative impact of a higher GHG concentration on welfare can be alleviated by higher consumption. If consumption and damages from global warming are complementary the reverse holds. Then, the marginal increase in welfare due to higher consumption is the higher the lower the GHG concentration and the planner will put less weight raising consumption in such a world. It should also be pointed out that the current-value Hamiltonian is strictly concave in its control variables but not necessarily in the control and state variables jointly and the maximized Hamiltonian is also not necessarily concave in the state variables. Therefore, the necessary conditions are not sufficient for a maximum and they only describe an extremal solution, a candidate for the optimum. Now we briefly discuss the necessary conditions for the first order state constraint (12). The state constraint satisfies the regularity condition ∂Ṡ/∂u = ∂Ṁ /∂u = 1 = 0. Hence, we can directly adjoin the state constraint by a scalar multiplier ν to the Hamiltonian and obtain the augmented Hamiltonian (cf. [11,19]): The adjoint equations (15) and (16) remain unchanged, while the adjoint equation (17) is replaced by (cf. [11,19]) The multiplier ν satisfies the complementarity conditions On a boundary arc with Thus the boundary control is given by In principle, this allows to compute the multiplier ν explicitly. Namely, on a boundary arc the following relation holds in view of (19) and (26): Differentiating both sides of this relation and using the adjoint equations (15), (16) and (24), one eventually gets a rather complicated expression ν = ν(X, λ) that we shall not write explicitly. Later in Section 4, we shall give an example with the numerically computed multiplier ν.

Steady state components.
Our first observation is that there is no steady state X * = (K * , R * , M * ) and λ * = (λ * K , λ * R , λ * M ) for the state and adjoint equations. We can only compute a steady state (K * , M * ) and (λ * K , λ * M ). To determine the stationary values (K * , M * ) and C * , we observe that u = 0 must hold in the long-run because the resource is finite. A zero long-run extraction rate implies that GHGs are obtained as M * = κM o . FromĊ = 0 in (22) we obtain where we used u = 0. It is noteworthy that the stationary state K * does not depend on the cost a of extracting one unit of the non-renewable resource. Finally, settingK = 0 leads to

HELMUT MAURER AND WILLI SEMMLER
The steady state shadow prices λ * K and λ * M are obtained as follows. Settingλ K = 0 in (15) and using M = κM o gives for the shadow price of GHGs From equation (15) we obtain for the shadow price of capital There is no defining equation to determine the steady state of the resource R. Hence, we can not expect a uniquely defined infinite-horizon solution. In the next section, we describe a method for designing suboptimal infinite-horizon controls.
3.3. Infinite-horizon optimal control. Since a stationary solution X * = (K * , R * , M * ) of the state equations does not exist, we can not expect a uniquely defined solution to the infinite-horizon control problem with the cost functional (13): In this section, we propose admissible solution candidates, where the trajectory on Thus, maximizing the infinite-horizon functional (13) amounts to maximizing the following cost functional on [0, T ] incorporating an additive salvage term: Numerical results for different terminal times T will be reported in Section 4.7.
3.4. Aims of the numerical explorations. Before we study the outcomes numerically for our model variant above, we want to note some differences to the optimization problem of the social planner and the laissez fair solutions; for details see Greiner et al. [9]. The rate of time preference of the agent in the laissez-faire economy need not coincide with that of the social planner. In the long-run, the optimum may differ for different discount rates. So we can compare a social planners problem with a problem of a market economy by comparing the solutions for different discount rates. If the time preference in the social optimum exceeds that of the market economy, capital, output and consumption in the social optimum are higher than in the market economy; for a theoretical proof see Greiner et al. [9].
But note it is obvious that the laissez fair economy and the social optimum do not coincide along the transition path unless the government corrects market failures. This difference in outcomes without government corrections is extensively discussed in Greiner et al. [9]. In a market economy the cost of emitting GHGs is neglected because this is an externality that is not considered by the households, unless the government intervenes and levies a tax on the use of non-renewable energy that generates the GHG emissions. The issue of how a laissez fair economy can be corrected by appropriate fiscal policies (tax rates and subsidies) will not pursued here further, see Greiner et al. [9].
The following points are interesting to pursue numerically. Is the resource completely used up for R(∞) = 0 or do we obtain R(∞) > 0, meaning that some non-renewable resource is left unextracted? In which cases do we get R(T ) > 0 for a finite time horizon decision model? To what extent do the results hinge upon the initial level of R(0) and K(0)? What is the role of the extraction cost of the non-renewable resource, and their efficiency in the production of output, relative to that of the renewable energy, the back stop technology. Further, what is the effect of state and control constraints on the solution of the model variants?
Another important issue is what the effects are when the phasing out of fossil fuel and phasing in of renewable energy will take place with some delay. One might consider different types of delays. Fossil fuel might be discovered but it is coming online, ready to be used for energy production, with a delay. There could also be a delay in the built up of the stock of atmospheric GHG when GHG is emitted, and there could be a delay in the phasing in of the green backstop technology. Wirl and Yegorov [28], for example, suggest that the phasing in of renewable energy will take place with a considerable delay. As an example of how such a delay can be treated we discuses the latter issue.
The stationary components for this set of parameters are computed from (27), (28) and (30) as Though it is of interest to study the sensitivity of the solution with respect to variations of all parameters, we investigate parametric sensitivity mainly with respect to the parameter a denoting the cost of extraction per unit. Seven case studies will be presented, in which we compute solutions for different initial and terminal conditions (11), for varying cost parameters a, for state constraints M (t) ≤ M max with various bounds M max , and for control delays in the production function and the dynamic equation for M . Moreover, in Section 4.7 we present infinite-horizon solutions for different values of T > 0 in the cost functional (31). Our numerical solution method of choice is the approach "first discretize then optimize". The optimal control problem is discretized on a fine grid (we mostly use N = 10000 grid points) using the implicit Euler scheme as integration method. This approach results in a high-dimensional nonlinear programming method which can be conveniently implemented by the Applied Modeling Programming Language AMPL [5] which is linked to the interior-point optimization solver IPOPT [27].
In the following computations, the initial condition for M and the terminal time T are fixed, and a control constraint for the consumption is imposed: Recall the remark after relations (19), (20) concerning the lower control bound in view of M (0) = M o = 1. Recall that X = (K, R, M ) ∈ IR 3 denotes the state variable and λ = (λ K , λ R , λ M ) ∈ IR 3 the adjoint variable. Henceforth, we shall denote by J = J(X, C, u) the optimal functional value.   Using the rather small cost factor a = 0.1 in the equationK = Y − C − δ K − a · u, the resource R is completely exhausted at time T . By increasing the cost factor a, the resource can be prevented from being completely exhausted. The computations show that R(T ) > 0 holds for a > a 0 ≈ 5.05. We list results for selected values of a using the abbreviation J = J(X, C, u):   Notice that the terminal constraint R(T ) ≥ 0.5 has the following effect: we have u(t) = 0 for t ≥ t 1 = 15.6, since R(t 1 ) = 0.5. The rather early stop of the extraction has the positive side-effect that the CO 2 concentration M approaches the equilibrium value M * = 2 at an early stage.
We also get nonzero values of the terminal resource R(T ) by modifying the dynamics (9) for K in such a way that the extraction cost depends on the size  of the resource R: Here, the adjoint equation for λ R in (16) has to be replaced bẏ A similar harvesting cost was used by Clark [4] in his famous model of optimal fishing. Since a 0 /R → ∞ for R → 0, we conclude that R(T ) > 0 must hold for any optimal solution. Let us report here the results for a 0 = 3, when R(T ) is free:   Note that the multiplier function ν(t) is discontinuous at the entry-time t 1 and exit-time t 2 of the respective boundary arc and, moreover, satisfies the strict complementarity condition ν(t) ≥ c for all t ∈ [0, T ] with a suitable constant c > 0.
It is remarkable that we can find a solution even for the state constraint M (t) ≤ M * = 2.0, where the bound is the stationary value.
and consider the dynamical system with a delay d M = 2 in the equation for the CO 2 concentration M , Note that we are interested in delay effects in our dynamical system for numerical reasons, but the delay effects in the CO 2 emission might also be justified in terms of incomplete knowledge of how CO 2 is built up in the atmosphere. A large but quite uncertain fraction of CO 2 emission is also absorbed by the ocean and forests. We just by-pass this uncertainty by using a delay effect of the CO 2 emission on the concentration of Green House Gas in the atmosphere. The initial function for the control u is given by Again, we consider the initial conditions are X(0) = (3, 3, 1) and prescribe the terminal conditions K(T ) = K * , R(T ) ≥ 0. Moreover, we impose the control and state constraints The control constraint is more restrictive than that in the Case Studies 1-5.
Necessary optimality conditions for optimal control problems with delays in the control and state variables may be found in Göllmann, Kern, Maurer [6] and Göllmann, Maurer [7]. We shall not discuss the necessary conditions here in detail and will only present some numerical results. As in the non-delayed case, the numerical approach is based on discretization and optimization methods. We obtain the numerical results   In this section, we consider the functional (31), For the terminal constraints K(T ) = K * , M (T ) = M * , we shall present solutions for various terminal times T . The welfare function value J ∞ (T, X, C, u) is increasing with increasing terminal time T . Hence, there does not exist a finite terminal time T for which this cost functional is maximal. Figure 15 displays the CO 2 concentration M (T ), resp., the terminal value R(T ) as function of the terminal time T ∈ [50, 250]. As one can observe the decision horizon T matters, in particular for the paths of resource R and the extraction rate u as well as for the amount of the resource left in situ.

Conclusion.
We extend an integrated assessment model of climate change by including alternative types of energy production. Damages from externalities of fossil energy appear in preferences and they interact multiplicatively with consumption in the household's welfare function. We use a numerical solution algorithm for finite time, called AMPL, to explore various options of how renewable energy might be phased in earlier before fossil energy is completely extracted. In the Case Studies 1-6, we presented numerical solutions for finite time decision problems for various initial and terminal conditions, under control and state constraints, and assuming delays in the control variable u. In all cases we also provided the adjoint variables and multipliers for the state constraints to make sure that the computed trajectories satisfy the necessary optimality conditions with high accuracy. The verification of second-order sufficient conditions via the Riccati approach discussed in Maurer, Pickenhain [20] and Malananowski, Maurer [18] is very involved and is beyond the scope of this paper.
The policy options we study indicate ways how the transition from non-renewable to renewable energy could be expedited. In our case studies we mainly focus on extraction cost of the non-renewable resource, the initial endowments of capital and nonrenewable resource, and the role of constraints and delays. If the extraction cost for fossil energy is low, fossil energy will be extracted until it is completely exhausted and with the consequence of higher carbon concentration in the atmosphere. This effect is reduced if constraints on extraction of fossil energy, constraints on consumption, carbon concentration and in particular higher extraction cost of fossil energy are introduced. A shorter decision horizon also appear to put constraints on the extraction of fossil energy and leaves more fuel in situ. On the other hand, delays in the extraction of non renewable energy and the build up of carbon concentration have only small effects on the paths of consumption and carbon concentrationusually at the beginning only. Further policy options such as a variation of private and public discount rates, variation in the efficiency of the production of renewable energy, different shapes of the damage function in the preferences as well as the effect of damages to production activities, due to climate change, could be explored in future work.