Optimal Control and the Fibonacci Sequence

Abstract

We bridge mathematical number theory with optimal control and show that a generalised Fibonacci sequence enters the control function of finite-horizon dynamic optimisation problems with one state and one control variable. In particular, we show that the recursive expression describing the first-order approximation of the control function can be written in terms of a generalised Fibonacci sequence when restricting the final state to equal the steady-state of the system. Further, by deriving the solution to this sequence, we are able to write the first-order approximation of optimal control explicitly. Our procedure is illustrated in an example often referred to as the Brock–Mirman economic growth model.

Appendix A

1.1 A.1 Proof: Theorem 2.1

We assume that standard regularity conditions of the optimal control problem hold; i.e., the criterion function f is sufficiently smooth and convex, and feasible policies lie within a compact and convex set. More specifically, we assume from here on that f is twice differentiable and that the Hessian of f is positive definite. With these premises, we derive the first-order approximation of the control function by applying the perturbation control technique, as outlined in, e.g., Sect. 4.6 in Lewis et al. [15].

The Lagrangian ($\mathcal{L}$) of the optimal control problem becomes

(20)

where μ _T+1 and λ _t+1 represent Lagrangian multipliers. A necessary condition for optimality is that the first variation of the Lagrangian is zero. In particular, the first variation of the Lagrangian evaluated for the steady-state is zero. An optimal control minimising the Lagrangian (20) can thus be approximated by incremental control minimising the second variation

where increments are made around the steady-state (i.e., $du_{t} := u_{t} -\bar{u}$ and $dx_{t}:=x_{t} - \bar{x}$) and where, e.g., the second partial derivative of f with respect to x _t , evaluated for the steady-state, is denoted by $f_{\bar{x}\bar{x}}$. This latter problem is recognised as the Lagrangian of the auxiliary discounted linear quadratic problem (DLQP):

(21)

where dλ _t+1 and dμ _T+1 represent the multipliers associated with the constraints (21). To simplify notation, we note the following identity (assuming ${f}_{\bar{u} \bar{u} }^{-1}$ exists):

Defining $d{\tilde{u}}_{t} :=(d{u}_{t} +{f}^{-1}_{\bar{u} \bar{u} } f_{\bar{x} \bar{u}} \,d{x}_{t})$ and ${\tilde{R}}:=(f_{\bar{x}\bar{x}} - f_{\bar{x} \bar{u}}{f}^{-1}_{\bar{u} \bar{u} } f_{\bar{x} \bar{u}} )$, the objective function in the DLQP is equivalent to

$$ \frac{1}{2} \sum^{T-1}_{t=0} \beta^{t} (dx_{t}{\tilde{R}} \,d{x}_{t} + d \tilde{u}_{t} \,{f}_{\bar{u} \bar{u} } \,d{\tilde{u}}_{t} ). $$

(22)

The constraint can be altered correspondingly. Inserting $d{u}_{t} =d{\tilde{u}}_{t} -{f}^{-1}_{\bar{u} \bar{u} } f_{\bar{x} \bar{u}} \,d{x}_{t}$ into (21) gives

$$ d{x}_{t+1} = {\bigl(A} -{B} {f}_{\bar{u} \bar{u} }^{-1} f_{\bar{x} \bar{u}}\bigr)\,d{x}_{t} + {B} \,d{\tilde{u}}_{t}. $$

(23)

To convert the problem to one without discounting, we define the variables ${\tilde{x}}_{t} :=\beta^{t/2} \,d{x}_{t}$ and ${\tilde{u}}_{t} := \beta^{t/2}\,d{\tilde{u}}_{t}$. Substituting these newly defined variables into (22) and (23) yields the linear quadratic problem (LQP):

(24)

where $\tilde{A} :=\beta^{1/2} ({A} -{B} {f}^{-1}_{\bar{u} \bar{u}} f_{\bar{x} \bar{u}} )$ and ${\tilde{B}} := \beta^{1/2} {B}$. Variables with a tilde are in the LQP that is thus transformed from the DLQP. As a result, the problem of finding the optimal plan that minimises the LQP is equivalent to finding the optimal plan that minimises the DLQP using the appropriate transformations. The LQP is well known, and its solution is given by⁷

(25)

This control function describes the optimal control for the LQP as a linear function of the state variable. The time-varying coefficient in front of the state variable consists of two parts. The first part $(L^{\mathbf{a}}_{t})$ is the feedback of an LQP when there is no restriction on the final state, i.e., $\tilde{x}_{T}$ is free to vary. It is determined by Eqs. (5) and (6). The last part of the feedback coefficient ($L^{\mathbf{b}}_{t}$) represents the linear part that ensures that the control function will drive the state to zero in the final time period, and it is determined by (7), (8), and (9).

We have now linked the first-order approximation of the control function and an LQP via a set of transformations. Having found a recursive solution to the linear quadratic problem we can derive the first-order approximation of the general problem by applying the set of transformations in reverse.

The optimal solution to the LQP is given by $\tilde{u}_{t} =-(L^{\mathbf{a}}_{t}- L^{\mathbf{b}}_{t})\tilde{x}_{t}$. Using the definitions ${\tilde{u} }_{t} := \beta^{t/2}\,d{\tilde{u}_{t}}$ and ${\tilde{x}}_{t} :=\beta^{t/2} \,d{x}_{t}$ yields

Further, substituting $d{\tilde{u}}_{t} :=(d{u}_{t} +{f}^{-1}_{\bar{u} \bar{u} } f_{\bar{x} \bar{u}} \,d{x}_{t})$ yields the optimal control of the DLQP:

Since increments are made around the steady-state, $du_{t} := u_{t} -\bar{u}$ and $dx_{t}:=x_{t} - \bar{x}$, the first-order approximated control function of the optimal control problem can be expressed by

(26)

where $L^{\mathbf{a}}_{t}$ and $L^{\mathbf{b}}_{t}$ are given by (5)–(9). This linearised control function ensures that the state reach the steady-state in the final period; i.e., restriction (3) holds also for this control function.⁸  □

1.2 A.2 Proof: Theorem 3.1

The first-order approximation (4) consists of two sequences: $L^{\mathbf{a}}_{t}$ and $L^{\mathbf{b}}_{t}$. We show the links between the generalised Fibonacci sequence and these sequences separately.

1.2.1 A.2.1 Fibonacci Sequence and Optimal Control: $L^{\mathbf{a}}_{t}$

First, we note that the ratio of Fibonacci numbers, $\mathcal{H}_{n}=\mathcal {F}_{n-1}/\mathcal{F}_{n}$, can also be generated by

(27)

with initial value $\mathcal{H}_{1}=0$. Further, combining (5) with (6), we can write $S_{t+1}={f}_{\bar{u} \bar{u}} \tilde{A} \tilde{B}^{-1} L^{\mathbf{a}}_{t+1}+\tilde{R}$, which when inserted into (5) yields

(28)

Comparing (27) with (28), we note that using the particular values $a = \tilde{B}$ and $b_{n+2}={f}_{\bar{u} \bar{u}}\tilde{R}^{-1}\tilde{A}^{2}$ when n is even and $b_{n+2}={f}_{\bar{u} \bar{u}} \tilde{R}^{-1}$ when n is odd makes (27) identical to the sequence of the transformed feedback (28) with an appropriate change of index. The sequence $\tilde{A}^{-1} L^{\mathbf{a}}_{t}$ runs backwards from an initial value at time t=T−1. If we make the index change n=2(T−t)−1, the sequence $\mathcal{H}_{n}=\mathcal{H}_{2(T-t)-1}$ begins at the initial value $\mathcal{H}_{1}=0$. Since from (5), the initial value of the feedback equation is zero, and consequently $\tilde{A}^{-1} L_{T-1} =0$, we have derived the following relationship:

(29)

1.2.2 A.2.2 Fibonacci Sequence and Optimal Control: $L^{\mathbf{b}}_{t}$

To derive the relationship between the second part of the control function ($L^{\mathbf{b}}_{t}$) and the generalised Fibonacci sequence, we note that the inverse of (27) can be written as

(30)

Multiplying the Riccati equation (6) by ($\tilde{B}\tilde{R}^{-1}$) yields

(31)

We note that the same choice of coefficients as in Appendix A.2.1 makes the sequence (30) identical to the sequence (31), i.e., $a= \tilde{B}$ and $b_{n+2}={f}_{\bar{u} \bar{u}}\tilde{R}^{-1}\tilde{A}^{2}$ when n is even, and $b_{n+2}={f}_{\bar{u} \bar{u}} \tilde{R}^{-1}$ when n is odd. The sequence ($\tilde{B}\tilde{R}^{-1}S_{t}$) runs backwards from time (t=T) with an initial condition, which follows from the Riccati equation $(\tilde{B}\tilde{R}^{-1}S_{T} )=0$. Since $\mathcal{F}_{0}/\mathcal{F}_{-1}=0$, we define $\mathcal{H}_{0}^{-1}:=0$, even though $\mathcal{H}_{0}$ is undefined. This gives the following relationship between the solution of the Riccati equation and the ratio of Fibonacci sequences, for 0≤t≤T:

(32)

Further, we note that from (8) and (9),

and hence, the initial condition $L_{T-1}^{\mathbf{b}}$ is, from (7),

$$ L_{T-1}^{\mathbf{b}}= \bigl( f_{\bar{u}\bar{u}}+\tilde{B}^{2}S_{T} \bigr)^{-1}\tilde{B}W_{T}P_{T-1}^{-1}W_{T-1}=- \frac{\tilde{A}\tilde{B}}{f_{\bar{u}\bar{u}}\frac{\tilde{B}^{2}}{f_{\bar{u}\bar{u}}}}=-\frac{\tilde{A}}{\tilde{B}}. $$

For k=0,1,2,3,… , we can rewrite the sequence of generalised Fibonacci numbers $( \mathcal{F}_{n} )$:

(33)

(34)

With these premises, we want to show that the second feedback coefficient also can be explicitly expressed in terms of generalised Fibonacci numbers; more specifically, we have that

(35)

(36)

(37)

To this end, we use the principle of induction. Bearing in mind that

we see that the initial conditions are satisfied because

(38)

In the following, we show that if the expressions (35)–(37) are true for k=p, then they are also true for k=p+1. Indeed, Eq. (8) with (29) yields that

where the last equality follows from (34). Moreover, Eq. (9) with (32) yields that

where we have used the relation (39), corresponding to d’Ocagne’s identity for regular Fibonacci numbers. Hence, expressions (35) and (36) follow according to the induction principle. Finally, expression (7) together with (32) for k=2,3,…,T, gives

where the last equality follows from (34).

In proving the explicit expression for P _T−k , we used the following identity:

$$ \mathcal{F}_{2k+2}\mathcal{F}_{2k-1}-\mathcal{F}_{2k} \mathcal{F}_{2k+1}= \tilde{A}\tilde{B} \biggl( \frac{\tilde{A}f_{\bar{u}\bar{u}}}{\tilde{R}} \biggr)^{2k-1},\quad k=0,1,2,\ldots. $$

(39)

This identity is also proved by induction. First, we note that the initial condition is satisfied because

$$ \mathcal{F}_{2}\mathcal{F}_{-1}-\mathcal{F}_{0} \mathcal {F}_{1}=\tilde{B}\frac{\tilde{R}}{f_{\bar{u}\bar{u}}}-0\cdot1= \tilde{A}\tilde{B} \biggl( \frac{\tilde{A}f_{\bar{u}\bar{u}}}{\tilde{R}} \biggr)^{-1}. $$

Now, let us assume that the identity is true for k=p, i.e.,

$$ \mathcal{F}_{2p+2}\mathcal{F}_{2p-1}-\mathcal{F}_{2p} \mathcal{F}_{2p+1}= \tilde{A}\tilde{B} \biggl( \frac{\tilde{A}f_{\bar{u}\bar{u}}}{\tilde{R}} \biggr)^{2p-1}. $$

The proof is complete if we can show that the expression also holds for k=p+1. Indeed,

where the last equality follows from the induction assumption. Changing the index, we have thus shown how the Fibonacci sequence enters the second feedback term:

(40)

 □

1.3 A.3 Proof: Corollary 3.1

Since

the control function

$$\tilde{u}_{T-k}=- \bigl( L_{T-k}^{\mathbf{a}}-L_{T-k}^{\mathbf {b}} \bigr) \tilde{x}_{T-k} $$

can be simplified when $\tilde{A}^{2}=1$. First, note that

If we let $\tilde{A}^{2}=1$, we then have that

$$L_{T-k}^{\mathbf{a}}-L_{T-k}^{\mathbf{b}}=\tilde{A} \frac{\mathcal {F}_{2k-1}^{2}}{\mathcal{F}_{2k-1}\mathcal{F}_{2k}}=\tilde{A}\mathcal{H}_{2k}, $$

by noting that

$$\mathcal{F}_{2k-1}^{2}-\mathcal{F}_{2k-2} \mathcal{F}_{2k}= \biggl( \frac{f_{\bar{u}\bar{u}}}{\tilde{R}} \biggr)^{2 ( k-1 ) }, \quad k=1,2,3,\ldots, $$

which follows from setting n=2k−1 in the following identity:

$$ \mathcal{F}_{n}^{2}-\mathcal{F}_{n-1} \mathcal{F}_{n+1}\mathcal{=} \biggl( -\frac{f_{\bar{u}\bar{u}}}{\tilde{R}} \biggr)^{n-1},\quad n=1,2,3,\ldots. $$

(41)

This identity is proved by induction. First, we note that the initial condition is satisfied because

$$\mathcal{F}_{1}^{2}-\mathcal{F}_{0} \mathcal{F}_{2}=1= \biggl( -\frac{f_{\bar{u}\bar{u}}}{\tilde{R}} \biggr)^{0}. $$

Now, let us assume that the identity is true for n=p, i.e.,

$$\mathcal{F}_{p}^{2}-\mathcal{F}_{p-1} \mathcal{F}_{p+1}= \biggl( -\frac{f_{ \bar{u}\bar{u}}}{\tilde{R}} \biggr)^{p-1}. $$

The proof is complete if we can show that the expression also holds for n=p+1. Indeed, using $\tilde{A}^{2}=1$, we get

where the penultimate equality follows from the induction assumption.

Remark A.1

Identity (41) is a generalisation of Cassini’s identity

$${F}_{n}^{2}-{F}_{n-1}{F}_{n+1}{=} ( -1 )^{n-1},\quad n=1,2,3,\ldots $$

for regular Fibonacci numbers.

Hence, in this special case, we have that

$$\tilde{u}_{T-k}=- \bigl( L_{T-k}^{\mathbf{a}}-L_{T-k}^{\mathbf {b}} \bigr) \tilde{x}_{T-k}=-\tilde{A}\mathcal{H}_{2k} \tilde{x}_{T-k}, $$

or

$$\tilde{u}_{t}=- \bigl( L_{t}^{\mathbf{a}}-L_{t}^{\mathbf{b}} \bigr) \tilde{x}_{t}=-\tilde{A}\mathcal{H}_{2(T-t)} \tilde{x}_{t}. $$

 □

1.4 A.4 Example: Narrative Details on the Brock–Mirman Model

The Brock–Mirman model considers a representative household maximising utility subject to economic constraints.⁹ In particular, it considers an economy where the total amount of goods (y _t ) is produced using capital (x _t ) as input in the production process; i.e.,

$$ y_t=\gamma x_{t}^\alpha, $$

(42)

where γ>0 and 0<α<1. In a closed economy, what is produced in a given year must be either consumed (c _t ) or invested (u _t ) as given by the national accounts identity:

$$ y_t = c_t + u_t. $$

(43)

Further, if we make the simplifying assumption that capital fully depreciates, the consecutive level of capital will equal current investments, i.e.,

$$ x_{t+1}=u_t. $$

(44)

Given an initial level of capital (x ₀), the objective of the representative household is to maximise a discounted (0<β<1) sum of utilities

$$ \sum^{T-1}_{t=0} \beta^{t} \ln(c_{t}), $$

(45)

subject to the three economic constraints (42)–(44) and subject to capital reaching the steady-state value in the final time period:

$$ x_T = \bar{x}. $$

(46)

The form of the Brock–Mirman model as given in the main text follows by inserting both the production function (42) and the national accounts identity (43) into the objective function (45). Further details on the Brock–Mirman model can be found in Sect. 3.1.2 in Ljungqvist and Sargent [12].

1.5 A.5 Example: Deriving the Steady-State

In this section, we derive the steady-state of the Brock–Mirman model. We define the Hamiltonian as

where λ _t+1 is the multiplier. The first-order conditions are¹⁰

Combining these first-order conditions and letting $c_{t}= \gamma x^{\alpha}_{t} - u_{t}$ yields the Euler–Lagrange equation:

For the steady-state, both the control and state remain unchanged, $\bar{c}=c_{t}=c_{t+1}$ and $\bar{x}=x_{t}=x_{t+1}$. The Euler equation can thus be solved to yield

Further, the steady-state levels of investment and consumption are given by

1.6 A.6 Example: Second Derivatives

In this section, we provide the second derivatives of the criterion function evaluated for the steady state. In particular, we have

From Appendix A.5, $\bar{c}=(1-\alpha)\alpha^{-1}$ when imposing the restrictions β=1 and γ=α ⁻¹. Also, inserting the normalisation $\bar{x} =1$ and α=1−ϕ ⁻¹ yields the results given in the main text:

To derive $f_{\bar{x} \bar{x}}$, we used the property

where the last equality follows from applying (11).

1.7 A.7 Example: Generalised Fibonacci Sequence

In this section, we illustrate that the Fibonacci sequence entering the control function of the Brock–Mirman model is the original Fibonacci sequence. The generalised Fibonacci sequence is in this example defined by

with the particular coefficients a=β ^1/2 B and $b_{n+2}={f}_{\bar{u} \bar{u}} (f_{\bar{x}\bar{x}} - f_{\bar{x} \bar{u}}{f}^{-1}_{\bar{u} \bar{u} } f_{\bar{x} \bar{u}} )^{-1}\times \beta({A} -{B} {f}^{-1}_{\bar{u} \bar{u}} f_{\bar{x} \bar{u}})^{2}$ when n is even and $b_{n+2}={f}_{\bar{u} \bar{u}} (f_{\bar{x}\bar{x}} - {f}^{-1}_{\bar{u} \bar{u} } f^{2}_{\bar{x} \bar{u}} )^{-1}$ when n is odd. It follows immediately that a=1. To find the expression for b _n+2, we need the second derivatives of the criterion function. From Appendix A.6, it follows that $f_{\bar{u} \bar{u}}^{-1}f_{\bar{x} \bar{u}}=-1$ and $f_{\bar{x} \bar{x}}-f_{\bar{u} \bar{u}}^{-1}f^{2}_{\bar{x} \bar{u}}=(1-\phi^{-1})$. Using the parameter values A=0 and β=B=1 yields b _n+2=1.

1.8 A.8 Section 4: Explicit Solutions of Odd- and Even-Indexed Fibonacci Sequences

Since (15) has constant coefficients, there is an explicit solution describing this sequence. We consider the following recurrence equation:

(47)

(48)

We note that (48) describes the sequence (15) when coefficients are matched, i.e.,

(49)

The general solution to the sequence g _n is well known and depends on whether the characteristic equation r ²=c ₁ r+c ₂ has two distinct real roots, one real double root or a pair of complex conjugate roots. Because of the assumption of a positive definite Hessian of f in the optimal control problem, only the real distinct roots are relevant, i.e., $r_{1,2}=(c_{1}\pm\sqrt{c_{1}^{2}+4c_{2}})/2$.¹¹ Given two initial values, the general solution is then given by

(50)

where G and K are constants to be determined from the initial conditions. These initial conditions depend on whether we are considering the odd- or even-indexed Fibonacci sequence.

We let $g_{n}^{e}$ denote the solution to (50) with initial values corresponding to even-indexed Fibonacci numbers. For example, we consider the initial value of the sequence $\mathcal{F}_{2(T-t-1)}$ when time is running backwards from t=T−1; i.e., $g^{e}_{0}=\mathcal{F}_{0}$ and $g^{e}_{2}=\mathcal{F}_{2}$. Given these initial conditions, solving for the constants G and K in (50) gives the following solution:¹²

In terms of the Fibonacci sequence, $g^{e}_{n}=\mathcal{F}_{n}$ when n is even. The explicit expressions for the even-indexed Fibonacci sequences entering the control function are then given by

(51)

(52)

Correspondingly, we let $g_{n}^{o}$ denote the solution to Eq. (50) with initial values corresponding to the initial value of the odd-indexed Fibonacci numbers, i.e., $g^{o}_{1}=\mathcal{F}_{1}$ and $g^{o}_{-1}=\mathcal{F}_{-1}$. Given these initial conditions, solving for the constants G and K in (50) gives the following solution:¹³

Since $g^{o}_{n}=\mathcal{F}_{n}$ when n is odd, the explicit solution of the odd-indexed Fibonacci sequence is then given by

(53)

1.9 A.9 Section 4: Both Roots are Real and Distinct

This section shows that both roots of the characteristic equation corresponding to the solution of every second generalised Fibonacci sequence are real and distinct because of the assumption of a positive definite Hessian of f. The general solution to the sequence

is well known and depends on whether the characteristic equation r ²=c ₁ r+c ₂ has two distinct real roots, one real double root or a pair of complex conjugate roots. Because of the assumption of a positive definite Hessian of the criterion function f in the optimal control problem, we show that only the real and distinct roots are relevant, i.e., $c_{1}^{2}+4c_{2}>0$. Indeed, given the expressions for c ₁ and c ₂, it follows that:

if $\frac{f_{\bar{u}\bar{u}}}{\tilde{R}}>0$. This holds because

$$\frac{f_{\bar{u}\bar{u}}}{\tilde{R}}=\frac{f_{\bar{u}\bar {u}}}{f_{\bar{x}\bar{x}}-f_{\bar{x}\bar{u}}f_{\bar{u}\bar{u}}^{-1}f_{\bar{x}\bar{u}}}=\frac{f_{\bar{u}\bar{u}}^{2}}{f_{\bar{u}\bar{u}}f_{\bar{x}\bar{x}}-f_{\bar{x}\bar{u}}^{2}}>0, $$

which is positive from the positive definiteness of the Hessian, implying

$$\left \vert \begin{array}{c@{\quad}c} f_{\bar{x}\bar{x}} & f_{\bar{x}\bar{u}} \\ f_{\bar{x}\bar{u}} & f_{\bar{u}\bar{u}}\end{array} \right \vert =f_{\bar{u}\bar{u}}f_{\bar{x}\bar{x}}-f_{\bar{x}\bar{u}}^{2}>0 \quad\text{and} \quad f_{\bar{u} \bar{u}}>0. $$

1.10 A.10 Section 4: The General Solution: g _n

The general solution to the difference equation

(54)

when both roots of the characteristic equation r ²=c ₁ r+c ₂ are real and distinct, is given by

(55)

where the constants G and K are determined by initial conditions. We consider the cases of even- and odd-indexed Fibonacci sequences separately, i.e., we find the sequences $g^{e}_{n}=\mathcal {F}_{n}$ when n is even and $g^{o}_{n}=\mathcal{F}_{n}$ when n is odd.¹⁴

1.10.1 A.10.1 Section 4: The Even-Indexed Sequence: $g^{e}_{n}$

From the generalised Fibonacci sequence, $g^{e}_{0}=\mathcal{F}_{0}=0$ and

which, when inserted into (54), gives the initial condition

where we have used the property c ₁=r ₁+r ₂. From the general solution (55), we get

Inserting this result when applying the second initial condition $g^{e}_{1}$ yields

Together, this implies

Using these results in the general solution (55) gives

Remark A.2

Note that $r_{1}^{2}-r_{2}^{2}=c_{1}\sqrt{c_{1}^{2}+4c_{2}}>0$.

1.10.2 A.10.2 Section 4: The Odd-Indexed Sequence: $g^{o}_{n}$

From the generalised Fibonacci sequence, $g^{o}_{1}=\mathcal{F}_{1}=1$ and $g^{o}_{-1}=\mathcal{F}_{-1}= \tilde{R}f^{-1}_{\bar{u}\bar{u}}$ , which gives

when inserted into (54) and applying the matched coefficient $c_{2}= f_{\bar{u} \bar{u}}\tilde{R}^{-1}\tilde{A}$. To determine G ^o and K ^o, we use the initial values $g^{o}_{0}$ and $g^{o}_{1}$. From the general solution (55), we get

or

(56)

From the other initial condition, we get

which, when inserting (56) and using the relation c ₁=r ₁+r ₂, yields

Inserting this result back into (56) gives

Using these results in the general solution (55) gives