A martingale approach for Pólya urn processes

This paper is devoted to a direct martingale approach for Pólya urn models asymptotic behaviour. A Pólya process is said to be small when the ratio of its replacement matrix eigenvalues is less than or equal to 1/2, otherwise it is called large. We find again some well-known results on the asymptotic behaviour for small and large urn processes. We also provide new almost sure properties for small urn processes.


Introduction
At the inital time n = 0, an urn is filled with α ≥ 0 red balls and β ≥ 0 white balls. Then, at any time n ≥ 1 one ball is drawn randomly from the urn and its color observed. If it is red it is then returned to the urn together with a additional red balls and b ≥ 0 white ones. If it is white it is then returned to the urn together with c ≥ 0 additional red balls and d white ones. The model corresponding replacement matrix is given, for a, b, c, d ∈ N, by R = a b c d .
The urn process is said to be balanced if the total number of balls added at each step is a constant, S = a + b = c + d ≥ 1. Thanks to the balance assumption, S is the maximum eigenvalue of R T . In fact, S is the Perron-Frobenius eigenvalue so it is simple. Moreover, the second eigenvalue of R T is given by m = a − c = d − b. Throughout the rest of this paper, our processes will be balanced and we shall denote σ = m/S < 1 the ratio of the two eigenvalues. It is straightforward that the respective eigenvectors of R T are given by We can rewrite R T under the following form Hereafter, let us define the process (U n ), the composition of the urn at time n, by U n = X n Y n and U 0 = α β where X n is the number of red balls and Y n is the number of white ones. Then, let τ = α+β ≥ 1 and τ n = τ +nS be the number of ball inside the urn at time n. In particular, one can observe that X n + Y n = τ n is a deterministic quantity.
The traditionnal Pólya urn model corresponds to the case where the replacement matrix R is diagonal, while the generalized Pólya urn model corresponds to the case where the replacement matrix R is not diagonal. The questions about the asymptotic behavior of (U n ) have been extensively studied, firstly by Freedman [9] and by many after, see for example [5,7,8,13,15,14]. We also refer the reader to Pouyanne's CIMPA summer school lectures 2014 [16] for a very comprehensive survey on Pólya urn processes that has been a great source of inspiration. The reader may notice that this paper is related to Bercu [4] on the elephant random walk. This is due to the paper of Baur and Bertoin [2] on the connection between elephant random walks and Pólya-type urns.
Our strategy is to use the martingale theory [6,11] in order to propose a direct proof of the asymptotic normality associated with (U n ). We also establish new refinements on the almost sure convergence of (U n ). The paper is organized as follows. In Section 2, we briefly present the traditional Pólya urn model, as well as the martingale related to this case. We establish the almost sure convergence and the asymptotic normality for this martingale. In Section 3, we present the generalized Pólya urn model with again the martingale related to this case, and we also give the main results for this model.
Hence, we first investigate small urn regime where σ ≤ 1/2 and we establish the almost sure convergence, the law of iterated logarithm and the quadratic strong law for (U n ).
The asymptotic normality of the urn composition is also provided. We finally study the large urn where σ > 1/2 and we prove the almost sure convergence as well as the mean square convergence of (U n ) to a non-degenerate random vector whose moments are given. The proofs are postponed to Sections 4 and 5.

Traditional Pólya urn model
This model corresponds to the case where the replacement matrix is diagonal It means that at any time n ≥ 1, one ball is drawn randomly from the urn, its color observed and it is then returned to the urn together with S ≥ 1 additional balls of the same color. Let us define the process (M n ) by M n = X n τ n and write where the conditional distribution of ε n+1 given the past up to time n is L(ε n+1 |F n ) = .
We now focus our attention on the asymptotic behavior of (M n ).
Theorem 2.1. The process (M n ) converges to a random variable M ∞ almost surely and in any L p for p ≥ 1. The limit M ∞ has a beta distribution, with parameters α S and β S . Remark 2.2. This results was first proved by Freedman, Theorem 2.2 in [9].
Our first new result on the gaussian fluctuation of (M n ) is as follows. Theorem 2.3. We have the following convergence in distribution

Gereralized Pólya urn model
This model corresponds to the case where the replacement matrix is not diagonal, Let us rewrite where the conditional distribution of ε n+1 given the past up to time n is L(ε n+1 |F n ) = B(τ −1 n X n ). We have Hence, we obtain that One can observe that ECP 25 (2020), paper 39.
Thanks to equation (3.1), we immediatly get that Hence, the sequence (M n ) is a locally bounded and square integrable martingale. We are now allowed to compute the quadratic variation of (M n ). First of all Moreover, Consequently, we obtain from (3.5) and (3.6) that The asymptotic behavior of (M n ) is closely related to the one of (w n ) with the following trichotomy: • The diffusive regime where σ < 1/2: the urn is said to be small and we have • The critical regime where σ = 1/2: the urn is said to be critically small and we .
• The superdiffusive regime where σ > 1/2: the urn is said to be large and we have Proposition 3.1. We have for small and large urns Proof. First of all, denote Λ n = I 2 + τ −1 n R T = P I 2 + τ −1 n D P −1 where I 2 is the identity matrix of order 2, and T n = n−1 k=0 Λ k . For any n ∈ N, T n is diagonalisable and

Small urns
The almost sure convergence of (U n ) for small urns is due to Janson, Theorem 3.16 in [13].

Theorem 3.2.
When the urn is small, σ < 1/2, we have the following convergence almost surely and in any L p , p ≥ 1.
Our new refinements on the almost sure rates of convergence are as follows.
In particular, Remark 3.6. An invariance principle for (X n ) was proved by Gouet, see Proposition 2.1 in [10].

Critically small urns
The almost sure convergence of (U n ) for critically small urns is again due to Janson, Theorem 3.16 in [13].
Once again, we have some refinements on the almost sure rates of convergence.

Large urns
The convergences of n −σ (U n − nv 1 ) to W v 2 first appeared in Pouyanne [15,Theorem 3.5]. The almost sure convergence of (U n ) for large urns is again due to Janson, Theorem 3.16 in [13]. The explicit calculations of the moments of W are new. almost surely and in L 2 , where W is a real-valued random variable and (3.24) ECP 25 (2020), paper 39.

Generalized urn model -small urns
Proof of Theorem 3.2. We denote the maximum eigenvalue of M n by λ max M n . We make use of the strong law of large numbers for martingales given e.g. by Theorem 4.3.15 of [6], that is for any γ > 0, where K is the covariance matrix from Theorem 3.5.. Therefore, we get from (4.1) that ECP 25 (2020), paper 39.
We now focus our attention on the law of iterated logarithm. We already saw that ∞ n=1 σ 4 n w 2 n < ∞.
Hence, it follows from the law of iterated logarithm for real martingales that first appeared in Stout [17,18], that for any u ∈ R d , lim sup Consequently, as M n (u) = σ n u, U n − E[U n ] , we obtain that lim sup In particular, for any vector u ∈ R 2 lim sup which together with (3.10) completes the proof of Theorem 3.3.

Generalized urn model -critically small urns
Proof of Theorem 3.7. The proof follows essentialy the same lines as the one of small urns in Theorem 3.2 and is left to reader.
Proof of Theorem 3.8. Again, the proof follows exactly the same lines as the one of small urns in Theorem 3.3 and shall not be explicited here.

Generalized urn model -large urns
Proof of Theorem 3.12. First, as Tr M n ≤ m 2 w n < ∞, we have that (M n ) converges almost surely to a random vector M v 2 , where M is a real-valued random variable and Hence, it follows from (3.4) that   Hence, we deduce (3.21) from (3.10), (4.2) and (4.3). The convergence in any L p for p ≥ 1 holds again by the same arguments as before. We now focus our attention on equation (3.22). We have from (3.10) and (4.2) that where the random variable W is given by We shall now proceed to the computation of E[W 2 ]. It is not hard to see that It follows from (3.10) that .
Consequently, we have which via (4.4) and (4.5) achieves the proof of Theorem 3.12.

Traditional urn model
Proof of Theorem 2.3. We shall make use of part (b) of Theorem 1 and Corollaries 1 and 2 from [12]. Let It is not hard to see that In order to use the Corollaries we need to verify that Consequently, the first condition of part (b) of Corollary 1 in [12] is satisfied with Let us now focus on the second condition of Corollary 1 in [12] and let ε > 0. On the one hand, we get that for all ε > 0 and we obtain condition 1-(b)(ii). Hereafter, it only remains to verify that the condition 2-(b) from Corollary 2 in [12] is satisfied. We easily get that is a martingale, the equation (5.1) proves that its bracket is convergent, which implies that the martingale is also convergent. This gives us Hence, the second condition of Corollary 1 in [12] is satisfied. Therefore we obtain that

Generalized urn model -small urns
Proof of Theorem 3.5. We shall make use of the central limit theorem for multivariate martingales given e.g. by Corollary 2.1.10 in [6]. First of all, we already saw from (4.1) that It only remains to show that Lindeberg's condition is satisfied, that is for all ε > 0, which ensures that Lindeberg's condition is satisfied. Consequently, we can conclude that M n √ w n L −→ n→∞ N 0, (1 − 2σ)K .
As M n = σ n U n −E[U n ] and √ nσ n is asymptotically equivalent to (1 − 2σ)w n , together with (3.10), we obtain that U n − nv 1 √ n L −→ n→∞ N 0, K .

Generalized urn model -critically small urns
Proof of Theorem 3.10. We shall also make use of the central limit theorem for multi- Once again, it only remains to show that Lindeberg's condition is satisfied, that is for all As in the proof of Theorem (3.5), we have Hence, Lindeberg's condition is satisfied and we find that M n √ w n L −→ n→∞ N 0, K .
As M n = σ n U n − E[U n ] and σ n √ n log n is asymptotically equivalent to √ w n , together with (3.10), we can conclude that