Constructive quadratic functional quantization and critical dimension

We propose a constructive proof for the sharp rate of optimal quadratic functional quantization and we tackle the asymptotics of the critical dimension for quadratic functional quantization of Gaussian stochastic processes as the quantization level goes to inﬁnity, i.e. the smallest dimensional truncation of an optimal quantization of the process which is “fully" quantized. We ﬁrst establish a lower bound for this critical dimension based on the regular variation index of the eigenvalues of the Karhunen-Loève expansion of the process. This lower bound is consistent with the commonly shared sharp rate conjecture (and supported by extensive numerical ex-periments). Moreover, we show that, conversely, optimized quadratic functional quantizations based on this critical dimension rate are always asymptotically optimal (strong admissibility result).


Introduction
The aim of this paper is two-fold: first we aim at providing a constructive proof of the sharp rate of optimal functional quantization in the quadratic case for (a wide class of) Gaussian processes or more generally of Gaussian random vectors X taking values in a separable Hilbert space H, (.|.) H . Secondly, we provide several results about the critical quantization dimension in this framework, especially a "sharp" asymptotic lower bound for the genuine critical dimension (sharp with respect to a conjecture supported by extensive numerical experiments carried out in [14] with the Brownian motion and the Brownian bridge) and the sharp asymptotics of the asymptotic critical dimension.
Before defining precisely what we mean by "constructive" and what the above two notions of critical dimension represent in an optimal functional quantization problem, let us briefly recall few basic facts on this theory. First, the L r -mean quantization error of a random variable X defined on a probability space (Ω, A, P) having a finite r th moment and taking values in a separable Hilbert space H is defined by e n,r (X) = inf min a∈α |X − a| H r , α ⊂ H, |α| ≤ n (1.1) where |α| denotes the cardinality of the set α, r ∈ (0, ∞) and . r denotes the L r (P)- where | . | H denotes the norm on the Hilbert space H. Any random vector which achieves the above minimum (there is always at least one) is called an (L r -)optimal n-quantization. One can show that such an optimal quantization is always of the form Y * = π α * (X) where π is a Borel projection on α * ,n = Y * (Ω) following the nearest neighbour rule. The subset α * ,n is called an optimal Voronoi n-quantizer (or optimal Voronoi quantizer at level n). By extension any random vector of the form π α (X) is called a Voronoi quantization whereas α is often called an n-quantizer (if |α| = n). The term Voronoi refers to the nearest neighbour projection. A sequence (α n ) n≥1 of n-quantizers is called asymptotically optimal if lim n min a∈αn |X − a| H r e n,r (X) = 1.
In the quadratic framework (r = 2), we will drop the subscript r for simplicity.
When H is an infinite dimensional separable Hilbert space (typically H is function space like L 2 ([0, T ], dt)), one often speaks of functional quantization.
The first problem of interest (beyond the existence of optimal n-quantizers for every n ∈ N) is the rate of decay of the mean (quadratic) quantization error. This sharp rate problem in a Hilbert setting has been solved for a wide class of H-valued Gaussian random vectors/processes X in [13] (see Theorem 2.2 below). To be precise, when the nonzero eigenvalues (λ X n ) n≥1 of the Karhunen-Loève (K-L) eigensystem (λ X n , e X n ) n≥1 of an H-valued random vector X ordered in a non-increasing order (each written as many times as is its multiplicity) read λ X n = ϕ(n) where ϕ is a non-increasing regularly varying function with index −b, b ≥ 1. Thus, we know that if X is a standard Brownian motion W on the interval [0, T ], then ϕ(n) = T 2 π 2 (n− 1 2 ) 2 so that b = 2 which implies lim n log n e n (W ) = √ π 2 T.
Note that the same sharp rate holds true for any Gaussian process whose parameter b is equal to 2, like the Ornstein-Uhlenbeck process. For a more general statement depending on the value of b ≥ −1, we refer to [13] where b is made explicit on many examples (Theorem 2.2 in the next section yields the general asymptotic result with an explicit limit, depending on b ≥ −1 and ϕ). This also applies to the fractional Brownian motion W H with Hurst constant H ∈ (0, 1) for which b = 2H + 1 since λ n ∼ κ H n −(1+2H) where κ H ∈ (0, +∞) is explicit, as established e.g. in [13]. The sharp rate for quadratic functional quantization can be solved by this approach for other processes like the fractional Ornstein-Uhlenbeck process, Gaussian sheets, etc.
Rather unexpectedly, the proof of this general theorem which was first stated in [13] is not constructive in the sense that it never needs nor exhibits any sequence of (asymptotically) optimal grids. The notion of critical dimension at level n, denoted d X n and referred to as genuine critical dimension in what follows, can be introduced in an intuitive way as follows d X n = min dim(span(α * ,n )), α * ,n optimal n-quantizer . As established in [12], the optimal quantizers at level n live in finite dimensional subspaces spanned by the (first components of the) Karhunen-Loève expansion of the Gaussian process of interest. So, in our Gaussian framework, the genuine critical dimension at level n is defined equivalently as the lowest dimension of such a vector subspace which contains an optimal n-quantizer, namely The "asymptotic critical dimension" corresponds to asymptotically optimal n-quantizers and will be precisely defined later on in Section 2. Like the mean quantization rate, the asymptotics of these critical dimensions are ruled by the rate of decay of the K-L eigenvalues. The genuine critical dimension can be considered, at level n, as the true dimension of the above infinite dimensional optimal quantization problem (1.1) and its specification as a kind of "dimension selection". As for the standard Brownian motion, the conjecture on the genuine critical dimension d W n reads lim n (log n) −1 d W n = 1.
and, more generally, There is a connection between the (quadratic) mean quantization error e n (X) and the critical dimension, still through the eigenvalues of the K-L expansion of X. Let (λ X n , e X n ) n≥1 be the K-L eigensystem of X, where the eigenvalues λ n are ordered in a non-increasing order. Then d X n is the lowest dimension satisfying This connection, originally established in [12], is briefly recalled in Section 2.3 as a starting point of our approach. Rather unexpectedly, the proof of the sharp rate for e n (X) in [13] does not provide asymptotic information on the critical dimension as n grows, at least in a straightforward way. In this paper, we fill these two gaps for this class of Gaussian processes: first, we propose a constructive proof of the sharp rate theorem for quantization error exhibiting asymptotically optimal grids and, secondly, we provide the first rigorous, though partial, results on the asymptotics of the critical dimension to our knowledge.
In particular we provide a first theoretical justification of the use of optimal quantizers at level n of the distribution 2 b log n k=1 N 0, λ X k to functionally quantize a Gaussian process X (provided its K-L system is explicit, in particular the eigenvectors). Such results are used to produce quadrature formulas to compute expectations of Lipschitz continuous or twice | . | L 2 T -differentiable functionals of Gaussian processes like the standard or the fractional Brownian motion, the Ornstein-Uhlenbeck process, etc, see e.g. [15]. It also has been used when X is diffusion solution to SDEs in the Stratonovich sense, see [15,16,4]: X is quantized by solutions of ODEs mimicking the SDE in which EJP 19 (2014), paper 50.
the Brownian W motion is replaced by its functional quantization. Several applications to the pricing of exotic path-dependent options based on this functional discretization method have been implemented. Functional quantization also has been used to as a stratification procedure for Monte Carlo simulation (see [11,3]).
The paper is organized as follows: in Section 2 we provide some more rigorous background on K-L expansions of Gaussian random vectors and functional quantization. Then, we state our main results by exhibiting a sequence of asymptotically optimal quantization grids and provide an asymptotic lower bound for the genuine critical dimension. In Section 3 and 4 we establish some upper and lower bounds respectively for the mean quantization error. Section 5 is devoted to the proofs and constructive aspects. We conclude by few numerical illustrations which support the conjecture. Our main tools, beyond discrete optimization techniques used for the upper bounds, are Shannon's source coding Theorem and the connection between mean quantization error and Shannon ε-entropy (or rate-distortion function, see [5]).
• Let (a n ) and (b n ) be two sequences of real numbers. a n ∼ b n if there exists a sequence (u n ) such that a n = u n b n and lim n u n = 1.
• o(1) denotes a sequence indexed by n ∈ N * going to 0 as n → +∞ (which may vary from line to line).

Karhunen-Loève expansion and main running assumption
Let X : (Ω, A, P) → H be a centered Gaussian random vector taking values in a separable Hilbert space (H, | . | H ) satisfying dim K X = +∞ where K X is the reproducing kernel Hilbert space of X.
A typical example is the case of a Gaussian stochastic process X = (X t ) t∈[0,T ] with continuous paths. Clearly, for such a process a.s. t → X t (ω) lies in L 2 T so that X can be see as a random vector taking values in L 2 Let (λ X k , e X k ) k≥1 be the orthonormal eigensystem of the (positive trace) covariance operator of X, also known as the Karhunen-Loève (K-L) orthonormal system of X. Since the sequence (λ X n ) n≥1 has only one limiting value, 0, one may assume without loss of generality that the K-L eigensystem is indexed so that the sequence of (nonzero) eigenvalues is non-increasing. To alleviate notations we will drop the dependency of the eigenvalues in X by simply noting λ n instead of λ X n . Throughout the paper, the main results are obtained under the following assumption about the K-L eigenvalues: There exists b ∈ [1, +∞) and a non-increasing function ϕ : (0, +∞) → (0, +∞) with regular variations at infinity of index −b (hence going to 0 at infinity) such that λ k = ϕ(k), k ≥ 1.

Optimal quadratic functional quantization
Let us introduce few additional notations. For every integer d ≥ 1, we denote by X (d) the H-orthogonal projection of X on the vector space span{e X 1 , . . . , e X d }, namely Note that one also has Finally we set, We know from [12] (see Proposition 2.1) that, for every n ∈ N * , the infimum in (2.1) holds as a minimum: there exists at least one optimal quantizer α * ,n which turns out to have full size n. Furthermore α * ,n lies in a finite dimensional space spanned by finitely many elements of the K-L basis. Now we are in position to come back to the genuine critical dimension d n = d X n defined by (1.2) and characterized in (1.3) as the smallest dimension of a vector subspace of span{e X n , n ≥ 1} in which an optimal n-quantizer lies. The sequence (d n ) n≥1 makes up a sequence satisfying e 2 n (X) = e 2 n (X, d n ). It is clear that d n goes to infinity, otherwise one could extract a subsequence d n such that d n ≤ d < +∞. If so, we would have e 2 n (X) ≥ k≥ d+1 λ k which contradicts the obvious fact that e 2 n (X) goes to zero as n goes to infinity (see e.g. [12]). This last claim is a consequence of the fact that, if (z n ) n≥1 is everywhere dense in H, then EJP 19 (2014), paper 50.
Otherwise very little is known on the sequence (d n ) n≥1 , in particular we do not know whether this sequence is monotone.
We will use the following easy lemma Proof. Let d ≤ d . Let α * ,n (d) be an optimal quadratic quantizer for X (d) of size (at most) n. It is clear that for every a ∈ α * ,n (d), As a consequence e 2 n (X (d ) ) ≤ e 2 n (X (d) ) + d k=d+1 λ k and one concludes by adding the tail k≥d +1 λ k .
It still holds as a conjecture that, under Assumption (R), whereas the sharp rate of optimal quadratic quantization has been elucidated for long in [13] (with several extension to more general Banach settings obtained ever since e.g. for L p ([0, T ], dt)-norms, 1 ≤ p ≤ +∞ and in an L r (P)-sense, see [6], etc).
Extensive computations carried out in [14] provide strong evidence that in fact, as concerns the standard Brownian motion and the Brownian bridge (which corresponds to b = 2), we even have that d n ∈ log n , log n .
These conjectures are also supported by results obtained for optimal "block quantization" with blocks of either constant or varying dimensions (see [13,14]).
The aims of this paper can now be summed up as follows : firstly to provide a constructive proof of the sharp rate theorem 2.2 recalled below (with, as a result, the exhibition of natural sequences of asymptotic quantizers), secondly to provide a partial answer to the above conjecture and finally to provide a complete answer to the "asymptotic" dimension problem.

Main result: constructive rates and critical dimension
Now we state our two mains results on the "constructive rates" and the critical dimension. While the genuine critical dimension d n is mostly of theoretical interest, for numerical purpose the "asymptotic critical dimension, precisely defined below, is more interesting. It corresponds to the lowest sequence of dimensions (δ n ) n≥1 which produce asymptotically optimal n-quantizers.
If the sequence of integers (δ n ) n≥1 goes to infinity and produces asymptotically optimal quantizers i.e. lim n e 2 n (X, δ n ) e 2 n (X) In fact this theorem can be reformulated equivalently in terms of critical dimension, at least when b > 1. To this end we introduce the following definitions. (b) If b > 1, an admissible sequence (δ n ) n≥1 is an asymptotic critical dimension sequence or strongly admissible for X if lim n ψ(δ n )e 2 n (X (δn) ) = 1.
Admissibility simply means that such a sequence produces asymptotic optimal quantizers in practice. For computational purpose, it is clear that the "lowest" choice δ n = 2 b log n seems the more appropriate. This is made more precise by the second definition in (b), even if it looks less intuitive.
As concerns asymptotic critical dimension (strong admissibility), we know from (2.3) that for every fixed dimension d ∈ N * , there is a balance in e 2 n (X, d) between the a priori quantized part e 2 n (X (d) ) and the tail of the series k≥d+1 λ k ) which represents the variance of the non-quantized part (or, equivalently, trivially quantized by 0). But for a fixed d this does not prejudge of what really occurs. However, an admissible sequence (δ n ) n≥1 being given, the smaller e 2 n (X (δn) ) is, the more quantized X (δn) is in practice.
As illustrated by Theorem 2.3, the definition of asymptotic critical dimension (strong admissibility) suggests that in that case X (δn) is "fully" quantized, at least asymptotically. This is confirmed by numerical computations (see [14] and [15] fro more insight on these numerical aspects). (a) If b > 1, As for the genuine critical dimension, we thus obtain the following lower bounds.
Following the definition of asymptotic critical dimension, it seems intuitive to guess that the sequence (d n ) n≥1 of genuine critical dimension is an asymptotic critical dimension sequence. In fact such a claim is just a reformulation of the conjecture (2.4). In more mathematical terms, it means that the conjecture is true if and only if lim n→+∞ ψ(d n )e 2 n (X (dn) ) = 1.

Upper bound (proofs)
Since we are trying to provide fairly a new constructive proof of Theorem 2.2 i.e. the sharp rate for quadratic functional quantization, we emphasize that we will not use any of its claims. In particular, we are not in position at this stage to claim that lim n ψ(log n)e 2 n (X) does exist. This is the reason why the claims in the proposition below involve lim sup n ψ(log n)e 2 n (X) which always exists (the same will be true with     First we need two lemmas devoted two block quantization and their critical dimension which are the key of the proof. For every integer d ≥ 1, we define set λ Proof. We introduce the (sub-optimal) d 0 -block product quantizer defined as follows where α ( ) ⊂ R d0 is an optimal quadratic quantizer of size (or at level) n of N 0; I d0 and Proj α ( ) : R d0 → α ( ) is a Borel nearest neighbour projection on α ( ) .

Elementary computations based on the Pythagoras theorem (see Lemma 4.2 in [13]) show that
The definition of C(d 0 ) completes the proof.
This optimal integer bit allocation has a formal almost optimal solution given by as suggested by considering the problem on (R + ) k instead of (N * ) k . This solution is admissible as soon as all the n s are nonzero or equivalently since they are non-increasing in as soon as n k ≥ 1.
Proof. (a) This follows from the fact that the sequence  log n d 0 and k n ≤ k n (d 0 ) and k n ≤ δn d0 so that by Lemma 3.3(c) we get as soon as n ≥ n d0 , e 2 n (X (δn) ) ≤ 4 Now, mimicking arguments in [13] involving regularly varying functions, namely ϕ, we One concludes by using (see [13]) that, owing to the converse of Shannon's source coding theorem, On the other hand (b) Assume first that lim n δn log n = κ ∈ (0, +∞). Owing to Lemma 3.2, we may assume as above that k n defined like in (a) satisfies k n ∼ κ log n where κ = κ ∧ 2 d0 . As b = 1, ψ(x) = 1 +∞ x ϕ(y)dy . It follows from Proposition 1.5.9b in [2] (applied with (y) = yϕ(y)) that ψ is a slowly varying function satisfying xϕ(x) = o(1/ψ(x)). Hence, we derive that On the other hand, EJP 19 (2014), paper 50. since ψ is slowly varying, and As a consequence lim sup n ψ(log n)e 2 n (X, δ n ) ≤ 1.
The extension to the general case lim inf n δn log n = κ ∈ (0, +∞) is straightforward up to the extraction of a subsequence.
Remark. Note that when b = 1, we do not need to let d 0 go to infinity. Since this rate is optimal (in view of Theorem 2.2), this means in particular that scalar product quantization (i.e. block quantization with blocks of size d 0 = 1) is asymptotically optimal.

Lower bound
We will rely on the famous notion in Information Theory, the Shannon ε-entropy (or rate-distortion function) of P (see [17]). Let P be a probability measure on H. For ε > 0, it is defined by where H(Q|P ⊗ Q 2 ) classically denotes the relative entropy (mutual information) dQ if Q is absolutely continuous with respect to P ⊗Q 2 , +∞ otherwise.
The simple converse part of Shannon's source coding theorem (see [1] Theorem 3.2.2, [7], p.163) says that the minimal number N (ε) of codewords needed in a codebook α such that E min a∈α X − a 2 ≤ ε 2 satisfies log N (ε) ≥ R(ε) so that, in particular R(e n (X)) ≤ log n.
By the definition of optimal quantization at level n, we have, as recalled above (see also [13]), ∀ n ≥ 1, R e n (X (d) ) ≤ log n.
Using the same trick (based on the sequence ( δ n ) n≥1 ) as in the former case), we derive similarly that, if lim sup n ψ(log n)e 2 n (X, δ n ) = lim sup n ψ(log n)e 2 n (X), then lim sup n ψ(log n) ψ(δ n ) ≤ 1 which is the announced result.

Proof of Theorem 2.3 (sharp rate and constructive aspects)
First we provide a proof of Theorem 2.2 based on the upper and lower bounds established in former sections and the following lemma (already established in [13] but reproduced here for the reader's convenience). Furthermore, it has to be noticed that it provides an easily tractable (and asymptotically optimal) lower bound for the quadratic quantization error, keeping in mind that the sequence (k n (1)) n≥1 is defined in Lemma 3.3.
Combining these two inequalities yields the announced result.
Then the three equivalences follow from (5.1), once noted again that

Proof of Theorem 2.5
Proof. (a) When b > 1, the direct claim on admissibility is a consequence of Proposition 4.3. The converse claim follows from Proposition 3.1(a) and Theorem 2.3.
As for strong admissibility, the direct claim is as follows: from the definition of strong admissibility, we get e n (X, δ n ) 2 ∼ e n (X) 2 (by admissibility) and e n (X, δ n ) 2 ∼ 1 ψ(δ n ) ∼ e n (X) 2 . Then comparing with the sharp rate from Theorem 2.3, which finally implies, having in mind that ψ is regularly varying with index b − 1, that δ n ∼ b 2 log n.
The converse claim is a consequence of The reverse inequality seems out of reach with the existing technology developed so far for functional quantization. However the strong admissibility result in Theorem 2.5(a) can be seen as an answer in the asymptotic sense since it shows that if lim n δ n log n = 2 b , then the resulting quadratic quantization error is asymptotically optimal and (asymptotically almost) all dimensions are used (strong admissibility).
This result is helpful from a numerical point of view since it shows that for the Brownian motion, the Brownian bridge or the Ornstein-Uhlenbeck process (and any Gaussian process for which b = 2, see below), considering a truncation at δ n = log n or δ n = log n is (at least) asymptotically optimal whatever the future of the sharper conjecture d n ∈ { log n , log n } could be.
For various other examples of families of processes satisfying Assumptions (R) (including multi-parameters processes like the Brownian sheet, we refer to [13]).  These values graphically fit with the monotony slope breaks. The graph in Figure 1 suggests, at this (low) range of the computation, that the limiting value for n → log(n)e 2 n (W ) is higher (≈ 0.22) than the theoretical one (≈ 0.2026).
This impression is misleading since further computations not reproduced here show that the sequence n → log(n)e 2 n (W ) starts to be slowly decreasing beyond n ≥ 1000. The value 0.22 seems to be a local maximum. For further details on these (highly time consuming) computations we refer to [14].
The quantization grids, computed during these numerical experiments by stochastic optimization methods (randomized Lloyd's procedure, Competitive Learning Vector Quantization algorithm) for n = 1 up to 10 4 for the standard Brownian motion (when T = 1), can be downloaded from the website www.quantize.maths-fi.com EJP 19 (2014), paper 50.