Near-optimal control for stochastic recursive problems

https://doi.org/10.1016/j.sysconle.2010.10.010Get rights and content

Abstract

It is well documented (e.g. Zhou (1998) [8]) that the near-optimal controls, as the alternative to the “exact” optimal controls, are of great importance for both the theoretical analysis and practical application purposes due to its nice structure and broad-range availability, feasibility as well as flexibility. However, the study of near-optimality on the stochastic recursive problems, to the best of our knowledge, is a totally unexplored area. Thus we aim to fill this gap in this paper. As the theoretical result, a necessary condition as well as a sufficient condition of near-optimality for stochastic recursive problems is derived by using Ekeland’s principle. Moreover, we work out an ε-optimal control example to shed light on the application of the theoretical result. Our work develops that of [8] but in a rather different backward stochastic differential equation (BSDE) context.

Introduction

Herein, we are primarily interested in the so-called “stochastic recursive optimal control problems” which can be illustrated by the following example arising from mathematical finance. Consider a financial market with two assets: one bond and one stock, and a “larger” investor who can decide at time t[0,T], the amount π(t) to invest in the stock with some initial endowment x0>0 and whose behavior can significantly influence the market price of the asset. It follows that his/her wealth process, denoted by x(), should satisfy some nonlinear stochastic differential equation (SDE) {dx(t)=b(t,x(t),π(t))dt+π(t)dW(t),x(0)=x0. Here W() is an R-valued standard Brownian motion on a certain probability space; b() is a suitable deterministic function. One interesting application of such nonlinear wealth equation can be found in Cvitanić and Ma [1] where the hedging option problem of a “larger” investor was studied. The case of constraints such as taxes can also be modeled as a special case of this setting (see El Karoui, Peng and Quenez [2]). We further suppose the investor can consume between 0 and T and denote c() the corresponding consumption rate process. Due to El Karoui et al. [2], the utility of the investor is a solution of {dy(t)=f(t,y(t),z(t),c(t))dtz(t)dW(t),y(T)=g(x(T)), where f() and g() are two appropriate deterministic functions. In the setup, the investor aims to choose a suitable pair (c̄(),π̄()) such that yc̄,π̄(0)=max(c(),π())yc,π(0). This formulates a (stochastic) recursive optimal control problem. In fact, Eq. (1) turns out to be a BSDE. The BSDE was originally proposed by Pardoux and Peng [3] in the general nonlinear case. The BSDE has been extensively applied in considerable different areas, especially in finance and economics. For instance, the celebrated Black–Scholes option pricing formula can be recovered by solving a special BSDE. For a comprehensive survey of BSDE theory, the interested reader may refer to the book by Ma and Yong [4] and the references therein.

In the literature [5], Duffie and Epstein independently established a stochastic differential recursive utility, which is a generalization of a standard additive utility with an instantaneous utility depending not only on an instantaneous consumption rate c(), but also on a future utility. El Karoui et al. [2] pointed out that this utility process can be characterized by a solution of some BSDE.

Due to its importance, the stochastic recursive control problems have been extensively studied in the literature like Peng [6], etc. It is remarkable that all these works typically focus on the study of the “exact” optimality whereas fairly little attention has been paid to the investigation of their near-optimal controls, to the best of our knowledge. However, it is really necessary and imminent to investigate the near-optimal controls of recursive optimal control problems due to their attractive merits from both theoretical and practical viewpoints. For example, the near-optimal controls always exist while the optimal controls only exist in relatively few cases; there are many near-optimal controls which can be selected appropriately for analysis and implementation.

Motivated by these points, this paper studies near-optimality for stochastic recursive control problems. However, it is necessary to point out that our study on the near-optimal controls is not a trivial and straightforward analog to that of the exact optimal controls. For instance, the classical maximum principle is essentially based on the fact that a minimum point of a function implies zero derivative at this point, while this is no longer the case for near-optimality, as a near-minimum of a function does not need to come out with a small derivative. Keep this in mind, we introduce some different techniques to solve these recursive optimal control problems. Our methods are mainly based on Ekeland’s principle. As the main result, a necessary condition as well as a sufficient condition of near-optimality to the aforementioned recursive controls is derived.

The structure of this paper is as follows. In Section 2, we formulate a recursive optimal control problem and introduce the definition of near-optimality. Sections 3 Necessary condition of near-optimality, 4 Sufficient condition of near-optimality are devoted to develop a necessary condition and a sufficient condition of near-optimality for the recursive optimal control problem. In Section 5, some concluding remarks are given.

Section snippets

Problem formulation and preliminaries

In this section, we outline the basic setup. We begin with a finite time horizon [0,1]. Let (Ω,F,(Ft)0t1,P) be a complete filtered probability space on which an Rd-valued standard Brownian motion (W()) is defined. It is also assumed that (Ft)0t1 is the natural filtration generated by (W()) and F=F1.

Throughout this paper, we make use of the notations: LF2(0,1;Rn):the set of all Rn-valued, Ft-adapted and square integrable processes ;fx:the gradient or Jacobian of a function f

Some prior estimates

This subsection is mainly devoted to investigate some prior estimates which play an important role in establishing our theoretical result.

Lemma 3.1

If Hypothesis (H1) holds, then there is a constant C such that for any v()UadEsup0t1|x(t)|2C,sup0t1E|y(t)|2C,Esup0t1|y(t)|2C,E01|z(t)|2dtC.

Proof

The first estimate is a well-known result, thus we focus on others. Squaring both sides of y(t)+t1z(s)dW(s)=Mx(1)+t1f(s,x(s),y(s),z(s),v(s))ds and using the fact that E[y(t)τt1z(s)dW(s)]=0, we get E|y(t)|2+E

Sufficient condition of near-optimality

In this section, we will show that, under certain convex conditions, the near-maximum condition of the Hamiltonian function in the integral form is sufficient for near-optimality. To start, we first introduce some definition and hypothesis.

Definition 4.1

Yong and Zhou [10]

Let XRn be a region and f:XR be a locally Lipschitz continuous function. The generalized gradient of f at xX, denoted by fx, is a set defined by f(x)={ξRn|ξ,ηlim¯yx,yX,h0f(y+hη)f(y)h}.

Hypothesis (H6)

For any t[0,1], (x,y,z)Rn×Rm×Rm×d, v and vU, the functions

Concluding remarks

The subject of near-optimality has been studied by only a few researchers, such as Zhou [8], Liu, Yin and Zhou [11], etc. Here, we particularly investigate the near-optimal controls to stochastic recursive optimization problems. This subsumes the study of near-optimality to the forward SDEs as its special cases. Meanwhile, as the classical methodologies fail to work in our setup, some new techniques are needed here to overcome this shortcoming. In this sense, our study is essentially different

Acknowledgements

The authors are grateful to the editor and an anonymous referee for their comments for improving the quality of this work.

References (11)

There are more references available in the full text version of this article.

Cited by (23)

View all citing articles on Scopus

Hui acknowledges the financial support from PolyU research account No. 1-ZV1X. Huang acknowledges the financial support from the Departmental General Research Foundation of Hong Kong Polytechnic University (No. G-YH42) and the RGC Earmarked Grant 500909. Li acknowledges the financial support from the RGC Earmarked Grants 524109 and 521610. Wang acknowledges the financial support from the National Science Foundation of China (10926098, 11001156), the Natural Science Foundation of Shandong Province, China (ZR2009AQ017), the Independent Innovation Foundation of Shandong University, China (2010GN063), and the startup foundation of doctoral research program of Shandong Normal University, China.

View full text