Relation between the rate of convergence of strong law of large numbers and the rate of concentration of Bayesian prior in game-theoretic probability

We study the behavior of the capital process of a continuous Bayesian mixture of fixed proportion betting strategies in the one-sided unbounded forecasting game in game-theoretic probability. We establish the relation between the rate of convergence of the strong law of large numbers in the self-normalized form and the rate of divergence to infinity of the prior density around the origin. In particular we present prior densities ensuring the validity of Erdos-Feller-Kolmogorov-Petrowsky law of the iterated logarithm.


Introduction
The most basic proof of the strong law of large numbers (SLLN) in Chapter 3 of [15] uses a discrete mixture of fixed proportion betting strategies. The mixture puts the weight 2 − j−1 on the points ±2 − j , j = 1, 2, . . . . The sum of weights on the interval [0, 2 −k ], k ≥ 1, is expressed as which is equal to the length of the interval [0, 1/2 k ]. Hence this mixture can be understood as the discrete approximation to the continuous uniform distribution over the interval [−1/2, 1/2]. In Chapter 3 of [15] this mixture is used only to prove the usual form of SLLN, and any discrete distribution having the origin as an accumulation point of mixture weights serves the same purpose. As we show in Section 4, the concentration of the mixture weights around the origin has the direct implication on the rate of convergence of SLLN forced by the mixture. In this paper we consider a continuous mixture in the integral form. Although a proof based on a discrete mixture is conceptually simpler, the integral in a continuous mixture is more convenient for analytic treatment. In fact in this paper we give a unified treatment covering the usual SLLN, the validity (i.e., the upper bound) of the usual law of iterated logarithm (LIL), and finally the validity of Erdős-Feller-Kolmogorov-Petrowsky (EFKP) LIL [13,Chapter 5.2].
Another feature of this paper is the self-normalized form in which we consider SLLN. Let S n = X 1 + · · · + X n , A 2 n = X 2 1 + · · · + X 2 n denote the sum of n random variables and and the sum of their squares, respectively. We compare S n to A 2 n , rather than to n. For example S n /n → 0 is the usual non-self-normalized SLLN, whereas S n /A 2 n → 0 is the SLLN in the self-normalized form. We often obtain cleaner statements and proofs in the self-normalized form. For instance, in measure-theoretic probability, Griffin and Kuelbs [7] showed the EFKP-LIL in the self-normalized form in i.i.d. case with some additional conditions. Later, Wang [18] and Csörgő et al. [3] eliminated some of the conditions. A similar argument in the self-normalized form can also be seen in game-theoretic probability as shown in [14]. Self-normalized processes including t-statistic are known for their statistical applications. In studying self-normalized processes, selfnormalized sums S n /A n are regarded as important and one of the reasons is that they have close relations to t-statistic; see, e.g., [5], [6]. For a survey of results concerning self-normalized processes, especially self-normalized sums, see [4], [16], and references in [14]. In the self-normalized form of SLLN we only consider paths of Reality's moves such that A ∞ = lim n A n = ∞. Hence the events considered in this paper are conditional events given A ∞ = ∞.
Yet another feature of this paper is that we mainly consider the one-sided unbounded forecasting game. We notice that the results on the bounded forecasting game can be derived from the results on the one-sided unbounded forecasting game. Also it is an interesting fact that in the one-sided unbounded forecasting game the usual non-self-normalized SLLN does not hold. We discuss this fact in Section 6. The importance of this game will be explained in Section 2 and Section 7.
We remark on the terminology used in this paper. We use "increasing" and "decreasing" in the weak sense, i.e. we simply write "increasing" instead of more accurate "(monotone) nondecreasing".
The organization of this paper is as follows. In Section 2 we introduce the protocol of the onesided unbounded forecasting game and relate it to betting on positive price processes. We also define Bayesian strategies with prior densities. In Section 3 we establish some preliminary results on the capital process of a Bayesian strategy. In Section 4 we prove a general inequality for the capital process of a Bayesian strategy and apply it to prove SLLN and the validity of LIL in the self-normalized form based on Bayesian strategies. In Section 5 we define two functionals which connect prior densities and functions of the upper class for EFKP-LIL. Based on these results, we give a proof of the validity of EFKP-LIL using a corresponding Bayesian strategy. In Section 6 we prove other basic properties of the one-sided unbounded forecasting game, including the rate of SLLN in the usual non-self-normalized form and the Reality's deterministic strategy against the usual form of SLLN. We end the paper with some discussions in Section 7.
2 The one-sided unbounded forecasting game and the Bayesian strategy In this paper we mainly consider the one-sided unbounded forecasting game defined as follows.
Reality announces x n ∈ [−1, ∞). K n = K n−1 + M n x n . Collateral Duties: Skeptic must keep K n nonnegative. Reality must keep K n from tending to infinity.
In game-theoretic probability the initial capital K 0 is usually standardized to be 1. However the value of the initial capital is not relevant in discussing whether Skeptic can force an event E or not. Also notation is sometimes simpler if we allow arbitrary positive value for K 0 .
In OUFG, Reality's move x n is unbounded on the positive side. On the other hand, in the bounded forecasting game (BFG) of Chapter 3 of [15], x n is restricted as |x n | ≤ 1. Since the Reality's move space is smaller in BFG than in OUFG, Reality is weaker against Skeptic in BFG than in OUFG. This implies that if Skeptic can force an event E in OUFG, then he can force E in BFG. In this sense the one-sided version of SLLN (cf. (3.9) in Lemma 3.3 of [15]) in OUFG is stronger than that in BFG. This is one of the reasons why we mainly consider OUFG in this paper.
Let n = M n /K n−1 denote the proportion of the capital Skeptic bets on the round n. The collateral duty of Skeptic is n ∈ [0, 1] in OUFG, whereas it is n ∈ [−1, 1] in BFG. This again reflects the fact that Skeptic is stronger in BFG than in OUFG.
OUFG is a natural protocol, when we consider betting on a positive price process {p n } of some risky financial assets. Let x n = p n p n−1 − 1 denote the return of the price process. Then x n ≥ −1 and there is no upper bound for the return x n if p n is allowed to take an arbitrary large value. Furthermore we can write K n = K n−1 + M n x n = K n−1 (1 + n x n ) as Hence (1 − n ) is the proportion of the current capital Skeptic keeps as cash, which does not change the value from round to round, and n is the proportion of the current capital Skeptic bets on the risky asset. When n ≡ is a constant, we call the strategy the constant-proportion betting strategy or thestrategy. The capital process of the -strategy is written as We now consider continuous mixture of -strategies where the initial capital is distributed according to the (unnormalized) prior density π ≥ 0 of an absolutely continuous finite measure on the unit interval [0, 1]: The betting proportion n of the Bayesian strategy with the prior density π is defined as ( Then the capital process K π n of this strategy is written as with the initial capital K π 0 = 1 0 π( )d . The equation (4) is checked by induction on n, since with n given in (3), we have A similar argument is also in [14]. One should notice that the equation (1) can be seen as one discretization of (4), where π is the uniform density on [−1/2, 1/2].
We are concerned with the rate of increase of π( ) as ↓ 0. Hence we require some regularity conditions on the behavior of π( ) around the origin. The following condition on π is convenient in discussing EFKP-LIL in Section 5.
For simplicity, we allow the case that the integral of π on [0, 1] is not 1, which does not cause a problem when considering limit theorems. Note that we are allowing ∞ = lim ↓0 π( ), but by the monotonicity and the integrability we are assuming 0 = lim ↓0 π( ). Assumption 2.1 holds for particular examples discussed in the next section. Concerning the condition on δ π in Assumption 2.1, Skeptic can always allocate the initial capital of δ π to the uniform prior of Section 4.2 to satisfy this condition.

Preliminary results
In this section we see the self-normalized form of SLLN in the one-sided unbounded forecasting game. Proposition 3.1. In OUFG, by any Bayesian strategy with π satisfying Assumption 2.1, Skeptic weakly forces We can prove this proposition by more or less the same way as in Lemma 3.3 of [15]. The difference is that the mixture we use here is not discrete but continuous. We will see later that we can show stronger forms in similar arguments.
We begin with the following simple lemma. The particular case C = 1 is used in Lemma 3.3 of [15]. We prove this lemma separately in a stronger form for later use.
Proof. Let Then g(0) = 0 and Hence Proof of Proposition 3.1. We use Lemma 3.2 with C = 1. Suppose that Reality has chosen a path such that A ∞ = ∞ and lim sup n S n /A 2 n > 0. Then there exists a sufficiently small 0 < δ < min( π , 1/2) such that S n > δA 2 n for infinitely many n. For such an n, Notice that the strategy and the capital process K π n do not depend on δ. Because lim n A n = ∞, we have lim sup n K π n = ∞.

Rate of convergence of SLLN implied by a Bayesian strategy
In this section we present results on the rate of convergence of SLLN by a Bayesian strategy, by establishing some lower bounds for the capital process. Note that since the Bayesian strategy already weakly forces (5), we only need to consider lower bounds when S n /A 2 n is sufficiently small. In Section 4.1 we present our first inequality, which will be used to show the rate of convergence of SLLN for the uniform prior in Section 4.2. Then we give a more refined inequality in Section 4.3, which will be used to show the rate of convergence of SLLN for the power prior (Section 4.4), for the prior ensuring the validity in the usual form of LIL (Section 4.5) and for the prior ensuring the validity of a typical form of EFKP-LIL (Section 4.6).

A lower bound for the capital process of a Bayesian strategy
Our first theorem of this paper bounds K π n from below as follows.
Proposition 4.2. In OUFG, by the uniform prior, Skeptic weakly forces Proof. Suppose that Reality chooses a path such that A ∞ = ∞, lim sup n S n /A 2 n ≤ 0 and lim sup Then for some small positive C, we have for infinitely many n. For such an n, Since lim n A n = ∞, we have lim sup n K π n = ∞. In Proposition 4.2, we considered continuous uniform mixture. However the same proof can be applied to discrete mixture setting. In particular the discrete mixture in the basic proof of SLLN in Chapter 3 of [15] achieves the same rate A 2 n ln A 2 n of (9). Note also that this rate is the same as the rate of SLLN of dynamic strategies studied in [9] and [8].
Then K π n ≥ δ π π 0 n i=1 (1 + x i )d and the result for the uniform prior applies to the right-hand side. Hence in considering other priors satisfying Assumption 2.1, we can only consider paths such that lim sup n S n / A 2 n ln A 2 n ≤ 1. In particular S 3 n = O((A 2 n ln A 2 n ) 3/2 ) and S 3 n /A 4 n → 0 as n → ∞. For such paths, we consider maximizing the right-hand side of (7) with respect to C. For the case S n /A n is large, we put C = A 2 n /S 2 n and obtain the following theorem.
Proof. Let C = A 2 n /S 2 n . By the assumption C < min(1/2, π ). Also S n A 2 Hence the conditions of Substituting the above inequalities into (7) we obtain (10).
Remark 4.4. In Section 5 we will multiply π( ) by a decreasing function c( ) and use c( )π( ) as the prior. Then c( )π( ) may not be increasing. However in view of the range of integration in (8), we can still bound the capital process by For the rest of this section we apply Theorem 4.3 to some typical nonuniform priors.
For this power prior, the following result holds. Proof. Consider the case S n > (1 + δ)(1 − a)A 2 n ln A 2 n for some δ > 0 for infinitely many n. Note that the function f (x) = x −a/2 e x/2 is increasing for sufficiently large x. Then for sufficiently large n,

Prior for the validity of LIL
Here we present a prior for the validity of LIL. Let 0 > 0 sufficiently small such that ln ln 1 is positive for ≤ 0 . Define π( ) = 1 ln 1 ln ln 1 2 , ∈ (0, 0 ).
This π is integrable around the origin.
Proof. Again we use the fact that f (x) = x −1/2 e x/2 is increasing for sufficiently large x. For S n > max( 2(1 + δ)A 2 n ln ln A 2 n , 1) for some δ > 0 and sufficiently large n, we have

Priors for the validity of typical EFKP-LIL
Here we generalize the prior (14) in view of EFKP-LIL. We give a more general and complete treatment of EFKP-LIL in Section 5. x.
The following prior density π( ) = 1 is integrable around the origin. We compare this with the bound A n ψ(A 2 n ), where For notational simplicity we take b ≥ 4 in (15), since the coefficient of ln k is 2 except for k = 3 in (16). Note that γ in (15) is multiplied by 2 in (16). This is needed, because unlike the usual LIL in (14), where we considered the ratio of S n to 2A 2 n ln ln A 2 n , in EFKP-LIL we need to consider the difference S n − A n ψ(A 2 n ). This difference corresponds to multiplication of π( ) by c( ) in the proof of Theorem 5.4.

Equivalence of prior densities and the upper class
The typical priors of the previous section suggest that higher concentration of the prior around the origin corresponds to a tighter convergence rate of SLLN. In particular, from the viewpoint of EFKP-LIL, it is of interest to establish the relation between priors and the function of the upper class. Also it is natural to conjecture that the rate of SLLN implied by a Bayesian strategy only depends on the rate of concentration of the prior around the origin. This idea can be clarified if we classify priors with the same rate of concentration into the same class. Let Π 0 denote the set of priors π satisfying Assumption 2.1 by Π 0 = {π | π satisfies Assumption 2.1}.

The upper class
In studying the validity of EFKP-LIL, the notion of upper class of functions is essential. A positive function ψ(λ) defined for λ > M ψ > 0 is said to belong to the upper class in OUFG if Skeptic can force the event For the terminology, see [13]. We characterize the upper class by an integral test: A typical function in the upper class is ψ given in (16). For convenience, we put the following regularity conditions on ψ.  (18) is finite, and 3. for λ > M ψ , we have λψ(λ)e −ψ(λ) 2 /2 > δ ψ .

the integral in
If lim λ↑∞ ψ(λ) is finite, then the integral in (18) for ψ satisfying Assumption 5.1. Furthermore the function xe −x 2 /2 is decreasing for x > 1 and converges to zero as x → ∞. Hence ψ(λ)e −ψ(λ) 2 /2 is eventually decreasing to zero. For EFKP-LIL, as the typical ψ in (16), we are mainly concerned with functions ψ, for which ψ(λ)e −ψ(λ) 2 /2 decreases to zero slower than any negative power of λ. Hence (19) is a mild regularity condition. We denote the set of functions ψ satisfying Assumption 5.1 by The goal is, when a ψ ∈ Ψ ∞ is given, to find a π ∈ Π 0 that is a witness of the ψ in the upper class. Now we define two functionals F : Ψ ∞ → Π 0 and G : Π 0 → Ψ ∞ as follows.
See Figure 1. These two functionals are "asymptotic inverse functionals" as we clarify in (27) and Theorem 5.8. Since we are interested in the case that is sufficiently small and λ is sufficiently large, for the rest of this paper we only consider λ that fulfills the condition −2 ln( 1 λ π( 1 λ )) ≥ 1. In fact, this condition holds true for sufficiently large λ because of the monotonicity of 1 λ π( 1 λ ) and lim λ↑∞ 1 λ π( 1 λ ) = 0. First we need to check that F[ψ] ∈ Π 0 and G[π] ∈ Ψ ∞ . For checking G[π] ∈ Ψ ∞ , we use the following relation.
Based on this lemma, we can consider the compositions F • G : Π 0 → Π 0 and G • F : Ψ ∞ → Ψ ∞ . By direct computation we obtain

Validity of EFKP-LIL via Bayesian strategy
Here we establish the validity of EFKP-LIL via a Bayesian strategy.
As a corollary to this theorem, we state the following statement of the validity of EFKP-LIL.
Proof. First suppose that ψ ∈ Ψ ∞ . Let π = F[ψ] and apply Theorem 5.4. Then Skeptic can weakly force for all sufficiently large λ. Hence S n − A n ψ(A 2 n ) ≤ 0 a.a. Next let ψ be a positive increasing function such that I(ψ) < ∞. Let ψ = min{ψ, √ 2 ln 2 n + 4 ln 3 n}. Notice that √ 2 ln 2 n + 4 ln 3 n belongs to the upper class. Since xe −x 2 /2 is decreasing for x > 1 and ψ ≤ √ 2 ln 2 n + 4 ln 3 n, we have Then, ψ satisfies Assumption 5.1. Hence, ψ belongs to the upper class and so does ψ. Finally, suppose that ψ is a positive increasing function, but I(ψ) = ∞. Then, in the fair-coin tossing game, Skeptic can force S n − √ nψ(n) ≥ 0 for infinitely many n. This is a corollary from the classical EFKP-LIL. This fact also follows from the main theorem in [14]. Thus, Reality can comply with this event with this restriction, which means that ψ does not belong to the upper class in OUFG.
Again it is easy to check that ∼ ∞ is an equivalence relation. In the following two lemmas we prove that the functionals F and G preserve the equivalence relations.
Since ψ 1 (λ) and ψ 2 (λ) diverge to ∞ as λ → ∞, for sufficiently small we have Dividing this by we obtain the lemma.
Therefore for sufficiently large λ Let Π 0 / ∼ 0 denote the set of equivalence classes in Π 0 with respect to ∼ 0 and define Ψ ∞ / ∼ ∞ similarly. Based on the above two lemmas and (25), (26) we have the following theorem.
Proof is obvious and omitted. The theorem says that functions in the upper class correspond to the prior densities, and the integrabilities of I(ψ) correspond to the integrabilities of densities.
6 Some other properties of the one-sided unbounded forecasting game In this paper we have been considering SLLN in the self-normalized form. However SLLN for OUFG in non-self-normalized form exhibits an interesting property, which we show in the following proposition. The definition of compliance can be found in [10] and [11].
Proposition 6.1. Let b n be a sequence of increasing positive reals such that lim n b n = ∞. In OUFG Skeptic can weakly force S n /b n → 0 if and only if n 1/b n < ∞.
Proof. Suppose that Z = n 1/b n < ∞. Consider Skeptic's strategy M n = 1/b n . Then Y n = Z + n i=1 M i x i is a nonnegative martingale and by the game-theoretic martingale convergence theorem (Lemma 4.5 of [15]) Skeptic can weakly force that Y n converges to a finite value. Then by Kronecker's lemma Skeptic can weakly force lim n S n b n = 0.
The converse part can be proved by a deterministic strategy of Reality complying with lim sup n |S n |/b n ≥ 1. Suppose n 1/b n = ∞. If b n < n − 1 for infinitely many n, then Reality chooses x n = −1 for all n, which complies with the desired property. Now, we assume that b n ≥ n − 1 for all but finitely many n. In each round Reality chooses either x n = −1 or x n = 2b n . Note that M n ≥ 0, otherwise, Skeptic would be bankrupt when Reality chooses x n large enough. Let p n = 1/(1 + 2b n ), We show that, for each n, at least one of x n ∈ {−1, 2b n } satisfies L n ≤ L n−1 . Suppose otherwise. Then Thus, −(1 − p n )M n + p n 2b n M n > 0, which implies This is a contradiction. Reality chooses her move so that L n ≤ L n−1 for all n. Then sup n K n ≤ 2. Furthermore, sup n n k=1 c k ≤ 2. If x n = 2b n for at most finitely many n, then there exists m such that x n = −1 for all n ≥ m, whence n k=1 c k = m−1 k=1 c k · n k=m 1 2(1−p n ) → ∞ because n p n = ∞. Hence, x n = 2b n for infinitely many n. For such an n that is large enough, we have S n =S n−1 + 2b n ≥ −(n − 1) + 2b n , S n b n ≥2 − n − 1 b n ≥ 1, which implies lim sup n S n /b n ≥ 1.
The intuition of p n is the probability of x n = 2b n so that the expectation is (−1) · (1 − p n ) + 2b n p n = 0. Then, n k=1 c k can be seen as a capital process in a game, and so is L. In this proof we constructed a particular deterministic strategy of Reality. General theory of constructing Reality's deterministic strategy complying with certain events is developed in [10] and [11].

Discussions
In this paper we gave a unified treatment of the rate of convergence of SLLN in terms of Bayesian strategies, including the validity of LIL. Concerning LIL we did not discuss the sharpness of the bound. In OUFG the sharpness does not hold, because Reality can simply take x n ≡ −1. For the sharpness we need some boundedness condition, such as the simplified predictably unbounded forecasting game (Section 5.1 of [15], [14]). Even with some boundedness conditions, current game-theoretic proofs of the sharpness are still complicated and the nature of the weights, involving also negative ones due to short selling of a strategy, does not seem to be clear. We hope that the unified treatment of this paper also helps to streamline proofs of the sharpness.
As we discussed in (2), a Bayesian strategy for OUFG can be understood as the portfolio of cash and one risky asset. In the literature on universal portfolio by Thomas Cover and other researchers (Chapter 16 of [2], [1], [12], [17]), Bayesian strategies for many risky assets are considered. They recommend power priors such as the Dirichlet prior, which corresponds to priors in Section 4.4. From the viewpoint of the rate of convergence of SLLN, we have shown that we should take a = 1 in (12) with additional multiplicative logarithmic terms. Hence our recommendations and those in universal portfolio literature seem to be different. This may be due to the difference of criteria for evaluating strategies. It is interesting to clarify these differences.