Random Periodic Processes, Periodic Measures and Ergodicity

Ergodicity of random dynamical systems in the random periodic regime where a periodic measure exists on a Polish space is obtained. In the Markovian random dynamical systems case, the idea of Poincar\'e sections is introduced. It is proved that if the $\tau$-periodic measure is PS-ergodic, then it is ergodic. Moreover, if the infinitesimal generator of the Markov semigroup has only $\{{2m\pi\over\tilde\tau}i\}_{m\in{\mathbb Z}}$ as simple eigenvalues on the imaginary axis, where $\tilde \tau={\tau\over k}$ for some $k\in{\mathbb N}\setminus\{0\}$, then the periodic measure is PS-ergodic. Furthermore, if the semigroup on Poincar\'e sections has spectral gap, then the periodic measure is PS-mixing. The distinction between random periodic and stationary regimes is given by a sufficient and necessary condition in terms of the spectral structure of the infinitesimal generators. In particular if the $\tau$-periodic measure is PS-mixing, then the infinitesimal generator of the Markovian semigroup has only $\{{2lm\pi\over\tau}i\}_{m\in{\mathbb Z}}$ for some $l\in{\mathbb N}\setminus\{0\}$ as simple eigenvalues on the imaginary axis, if and only if the minimal period of the periodic measure is $\tilde\tau={\tau\over k}$ for some $k\in{\mathbb N}\setminus\{0\}$. The ergodicity is extended to the periodic stochastic semi-flows that is lifted up to a cocycle on the cylinder $[0,\tau)\times{\mathbb X}$. It is proved that the infinitesimal generator of the Markovian semigroup of the lifted periodic diffusion process on a finite dimensional Euclidian space has the desired spectral property if the generator of the diffusion has only one simple eigenvalue $0$ on the imaginary axis for each fixed time. The"equivalence"of random periodic processes and periodic measures is established. The strong law of large numbers is also proved.


Introduction
The idea regarding stochastic differential equations as random dynamical systems went back to late 1970's with a number of seminal works ( [3], [4], [15], [27], [28], [34] etc). Later this was developed to include stochastic partial differential equations in [20], [35], [22], [24]. With these foundational works in hands, similar to the case of deterministic dynamical systems, to study the long time behaviour lies in the centre of the theory of random dynamical systems.
Fixed points and periodic paths are two basic notions in the theory of dynamical systems capturing their long time behaviour and equilibrium. The concept of the stationary solution has been known for some time and is the corresponding notion of fixed points in the stochastic counterpart. Their existence has been subject to intensive studies ( [14], [21], [31], [33], [38], [40], [44]). Moreover, it is well known that a stationary solution gives rise to the existence of an invariant measure ( [1]). The converse part is not true in general ( [36]). But in an enlarged probability space, an invariant measure is a random Dirac measure, so there exists a pathwise stationary solution ( [1]). The study of invariant measures has been one of the central problems in the areas of ergodic theory and stochastic partial differential equations in the last few decades ( [5], [6], [13], [21], [25], [29] to name but a few). The notion of periodic solution is another key concept in the theory of dynamical systems, and has occupied a central space in the theory of dynamical systems since Poincaré's pioneering work. A notion of random periodic solutions should play a similar role in the theory of random dynamical systems. Periodic phenomena exist in many real world problems e.g. biology, economics, chemical reactions, climate dynamics etc. But, by nature, many real world systems are very often subject to the influence of internal or external randomness.
Periodicity and randomness may often mix together. For instance, the maximum daily temperature in any particular region is a random process, however, it certainly has periodic nature driven by the divine clock due to the revolution of the earth around the sun.
Physicists have attempted to study random perturbations to periodic solutions for some time by considering a first linear approximation or asymptotic expansions in small noise regime, e.g. see [41], [43]. One of the obstacles to make a systematic progress was the lack of a rigorous mathematical definition of random periodic solution and appropriate mathematical tools. For a random path with some periodic nature, it was not clear what a mathematically correct relation between the random position Y (t, ω) at time t and Y (t + τ, ω) at time t + τ after a period τ should be. In the meantime, as Y (t, ω) is a true path or solution, so it is not true in general that Y (t, ω) = Y (t + τ, ω). The approach in [43] was to seek Y (t + τ, ω) in a neighbourhood of Y (t, ω) which was applicable to small noise perturbations.
New observation was that for each fixed t ∈ I + , {Y (t + kτ, ω)} k∈N should be a random stationary solution of the discretised random dynamical system Φ(kτ, ω) ( [18], [19], [30], [45]). Here Φ : I + ×Ω×X → X is a random dynamical system over a metric dynamical system (Ω, F , P, (θ t ) t∈I ). This then led to the rigorous definition of pathwise random periodicity Y (t + τ, ω) = Y (t, θ(τ )ω) and existence of random periodic solutions for cocycles and stochastic semiflows defined e.g. by SDEs and SPDEs. The concept of pathwise random periodic paths (solutions) is the stochastic counterpart of periodic paths in the theory of dynamical systems. It gives rigour and a clearer understanding to physically interesting problems of certain random phenomena with a periodic nature. On the other hand, similar to the stationary solution, the random periodic path also represents a long time limit of the underlying random dynamical system, therefore is one of the basic objects in its geometric structure. The existence of random periodic solutions of various stochastic systems and stochastic (partial) differential equations was obtained in [18], [19], [30], [45]. More recently there have been more progresses on the study of random with periodic forcings. This was motivated in the context of studying the climate change problem when the seasonal cycle is taken into considerations ( [23], [32]), in the context of the Brussellator arising in chemical reactions ( [39]), and Ornstein-Uhlenbeck processes ( [9]).
In this paper, we will first discuss the relation between random periodic paths and periodic measures. In particular, we will prove that a random periodic path of a random dynamical system gives rise to a periodic measure of the skew product flow in the product measurable space and Markov semigroup in phase space respectively. For a cocycle random dynamical system, we will construct an invariant measure as the time average of the periodic measure in a time interval of one period. Conversely, in general a periodic measure cannot give a random periodic solution in the original probability space. We will construct a new probability space which is enlarged by adding trajectories of the random dynamical systems on the lifted phase space to be part of the new noise paths. We can then extend the random dynamical systems naturally over the enlarged probability space and proved that the pull-back of the random dynamical systems is a random periodic path. These results reveal the natural relation between pathwise random periodicity and periodicity in the sense of distributions. In fact, one can prove that the law of the random periodic path is the very periodic measure. For this reason, we call the random periodic path the random periodic process in the future. Both cocycles and semi-flows can possess random periodic processes (see [18], [19], [45]). We will discuss these two cases separately in the following sections.
One importance of the relationship of the random periodic processes and periodic measures is to enable us to establish the strong law of large numbers (SLLN). The SLLN was known for sequences of independent and identically distributed random variables (Kolmogorov's SLLN theorem) and stationary processes/invariant measures (Birkhoff's ergodic theory). We will add in this investigation to prove that the SLLN holds for random periodic processes/periodic measures. We would like to mention that in a different direction, the SLLN was also investigated recently in [7] for independent and identically distributed random variables in the sense of non-additive probability ( [37]). The SLLN result proved here implies that if Y (s, ω), s ∈ R is a random periodic solution of a Markov dynamical system on a polish space X over a metric dynamical system (Ω, F , P, (θ t ) t∈R ), with its law L(Y (s)) = ρ s , where there exists a constant τ > 0 such that Y (s + τ, ω) = Y (s, θ τ ω) for all ω ∈ Ω and ρ s+τ = ρ s , then under an exponential convergence condition of the transition semigroup to the periodic measure (see Condition (EC) in section 2), we have as T → ∞, for any f ∈ L 1 (X, B X ,ρ(dx)). Hereρ = 1 τ τ 0 ρ s ds.

The cocycle case
Let X be a Polish space and B X be its Borel σ-algebra, a common notation I be either R, two-sided continuous time or {0, ±1, ±2, · · ·}, two-sided discrete time, two different scenarios of time. Denote by a common notation ν the Lebesgue measure in the continuous time case or the counting measure in the discrete time case on I. We use I + (I − ) to denote one-sided time, i.e. I + = R + (I − = R − ), one-sided continuous time; and I + = {0, 1, 2, · · ·} (I − = {0, −1, −2, · · ·}), one-sided discrete time.
In this section, we consider a measurable cocycle random dynamical system Φ on (X, B X ) over a metric dynamical systems (Ω, F , P, (θ(t)) t∈I + ) with a one-sided time set I + , Φ : I + × Ω × X → X. It is (B I + ⊗ F ⊗ B X , B X )-measurable and satisfies the cocycle property: for all s, t ∈ I + and ω ∈ Ω. The map θ(t) : Ω → Ω is P -measure preserving and measurably invertible. Therefore it can be extended to I − as well by setting θ(t) = θ(−t) −1 when t ∈ I − . There is no need to require the map Φ(t, ω) : X → X to be invertible, which enables the work to be applicable to both SDEs and SPDEs. Therefore we can consider a cocycle measurable random dynamical system Φ on (X, B X ) over a metric dynamical systems (Ω, F , P, (θ(t)) t∈I ) on a one-sided time set I + .
First recall the definition of random periodic paths (solutions) given in [18], [19]. The definition of stationary solutions was well known. But we include it here to make a comparison between the concepts of random periodic paths and stationary solutions. The same remark also applies to invariant measures in Definitions 2.2 and 2.5.
Definition 2.1 A random periodic path of period τ of the random dynamical system Φ : I + ×Ω ×X → X is an F -measurable map Y : I × Ω → X such that for any t ∈ I + , s ∈ I and ω ∈ Ω, It is a stationary solution of Φ if Y (s, ω) = Y (0, θ(s)ω) =: Y 0 (ω) for all s ∈ I, i.e. Y 0 : Ω → X is a stationary solution if for any t ∈ I + and ω ∈ Ω, 3) The first part of the definition of the random periodic path suggests that a random periodic path Y (t, ω) is indeed a pathwise trajectory of the random dynamical system. The second part of the definition says that it has some periodicity. But it is different from a periodic path in the deterministic case, Y (t + τ, ω) is not equal to Y (t, ω), but Y (t, θ(τ )ω). We call this random periodicity. Starting at Y (t, ω), after a period τ , trajectory does not return to Y (t, ω), but to Y (t, ·) with different realisation θ(τ )ω. So it is neither completely random, nor completely periodic, but a mix of randomness and periodicity.
(2.4) Therefore φ is a periodic function and the graph is a closed curve. It is easy to see from the first formula in (2.4) that L ω is an invariant set, i.e.
Φ(t, ω)L ω = L θ(t)ω , t ∈ I + . But needless to say that random periodic solution gives more detailed information about the dynamics of the random dynamical system than a general invariant set.
Unlike the periodic solution of deterministic dynamical systems, the random dynamical system does not follow the closed curve, but move from one closed curved to another when time evolves. This is fundamentally different from the deterministic case, which makes it hard to study. However, this natural definition in random case makes it possible to gain new understanding of random phenomena with some periodic nature, where strict deterministic periodicity is not applicable.
First we prove that the random Dirac measure with the support on sections of the random periodic curve L ω is the periodic measure of the random dynamical system and its time average is an invariant measure. To make this clear, we consider a standard product measurable space (Ω,F ) = (Ω × X, F ⊗ B X ) and the skew-product of the metric dynamical system (Ω, F , P, (θ(t)) t∈I ) and the cocycle Φ(t, ω) Definition 2.2 A map µ : I → P P (Ω × X) is called a periodic probability measure of period τ on (Ω × X, F ⊗ B X ) for the random dynamical system Φ if µ τ +s = µ s andΘ(t)µ s = µ t+s , f or any t ∈ I + , s ∈ I. (2.7) It is an invariant measure if it also satisfies µ s = µ 0 for any s ∈ I i.e. µ 0 is an invariant measure of Φ if µ 0 ∈ P P (Ω × X) andΘ (t)µ 0 = µ 0 , f or any t ∈ I + . (2.8) 3 If a random dynamical system Φ : I + × Ω × X → X has a random periodic path Y : I × Ω → X, it has a periodic measure on (Ω × X, F ⊗ B X ) µ : I → P P (Ω × X) given by where A ω is the ω-section of A. Moreover, the time average of the periodic measure defined bȳ is an invariant measure of Φ whose random factorisation has the support L ω defined by (2.5).
Proof: It is obvious that P is the marginal measure of µ s on (Ω, F ), so µ s ∈ P P (Ω × X). To check (2.7), first note for t ∈ I + ,Θ(t) −1 (A) = {(ω, x) : (θ(t)ω, Φ(t, ω)x) ∈ A}. Then it is easy to see that for Second from the definition of random periodic path and the probability preserving property of θ, we have Thus µ s , s ∈ I defined by (2.7) is a periodic measure as claimed in the theorem. To seeμ defined by (2.10) is an invariant measure, note for any A ∈ F ⊗ B X and t ∈ I + ∩ [0, τ ), by what we have proved A similar argument can be used to prove that any t ∈ I + , (2.14) Thusμ is an invariant measure. To see its support, by (2.10), (2.9) and Fubini's Theorem, for any This leads to its factorisation given by It follows easily that L ω is the support of (μ) ω . ‡ ‡ Remark 2.4 For a random periodic path Y , it is easy to see that the factorization of µ s defined in Theorem 2.3 is

15)
and satisfies Now consider a Markovian cocycle random dynamical system Φ on a filtered dynamical system (Ω, F , P, (θ t ) t∈I , (F t s ) s≤t ), i.e. for any s, t, u ∈ I, s ≤ t, θ −1 u F t s = F t+u s+u and for any t ∈ I + , Φ(t, ·) is measurable with respect to F t 0 . We also assume the random periodic solution Y (s) is adapted, that is to say that for each s ∈ I, Y (s, ·) is measurable with respect to F s −∞ := ∨ r≤s F s r . Denote the transition probability of Markovian process Φ(t, ω)x on the Polish space X with Borel σ-field B X by (c.f. Arnold [1], Da Prato and Zabczyk [13]) P (t, x, B) = P ({ω : Φ(t, ω)x ∈ B}), t ∈ I + , B ∈ B X , and for any probability measure ρ on (X, B X ), define Definition 2.5 A measure function ρ : I → P(X) is called a periodic measure of period τ on the phase space (X, B X ) for the Markovian random dynamical systems Φ if it satisfies ρ s+τ = ρ s and ρ t+s = P * (t)ρ s , s ∈ I, t ∈ I + . (2.17) It is called an invariant measure if it also satisfies ρ s = ρ 0 for all s ∈ I, i.e. ρ 0 is an invariant measure for the Markovian random dynamical system Φ if ρ 0 = P * (t)ρ 0 , for all t ∈ I + .
(2.18) Theorem 2.6 Assume the Markovian cocycle Φ : I + × Ω × X → X has an adapted random periodic path Y : I × Ω → X. Then the measure function ρ : I → P(X) defined by which is the law of the random periodic path Y , is a periodic measure of Φ on (X, B X ). Its time averagē ρ over a time interval of exactly one period defined bȳ is an invariant measure and satisfies that for any B ∈ B X , t ∈ Ī Proof: Firstly it is easy to see from the definition of random periodic path that for any B ∈ B X , . Therefore for any B ∈ B X , t ∈ I + , by measure preserving property of θ, independency of Φ(t, θ(s)ω) and F s −∞ , Therefore ρ satisfies Definition 2.5 so is a periodic measure on (X, B X ). To prove the second part of the theorem, similar to the computation in (2.13), we have for any t ∈ [0, τ ) ∩ I + , and by using Fubini Theorem, It then follows easily thatρ is an invariant measure of Φ satisfying (2.18). To prove the last part of the theorem, from (2.20), (2.19), and using Fubini's Theorem, we know for any B ∈ B X , However, sinceρ is an invariant measure, so from (2.22) we know that for any t ∈ I + For t ∈ I − , It is easy to verify P * (−t)ρ s+t = ρ s and thereforē So we can see that (2.21) is true for any t ∈ I. ‡ ‡ Identity (2.21) says that the expected time spent inside a Borel set by the random periodic path over a time interval of exactly one period starting at any time is invariant, i.e. independent of the starting time. This shows that the random periodicity of a random periodic path by means of invariant measures. In the following we will push the above observation further to study the strong law of large numbers which says on long run, the average time that the random periodic path spends on on a Borel set B over one period is equal toρ(B) a.s. In order to prove this result, we use a weak law of large numbers (WLLN) for sequences of dependent random variables with long range decay covariance. This result may not be new and the proof is quite easy. But we can't find any reference. So we include it here for convenience without claiming its originality.
Lemma 2.7 (WLLN) Let X n be a sequence of identical distributed variables and S n := n i=1 X i denote the n-th partial sum of X n . Assume that |cov( Then for any ǫ > 0, Proof: For any ǫ > 0 and n ∈ N , by Chebyshev's inequality, we have We can explicitly compute n i,j=1 α |i−j| as follows: Therefore, Instead of considering the whole interval [0, T ], we can consider a "window" in the first period [0, τ ), and identical copies of the window in the subsequent periods. We will prove the SLLN in those windows. The case of continuous internal [0, T ] (T → ∞) is a special case of the results that we will prove here. For this, let F 0 ⊂ [0, τ ) ∩ I be a given Borel set on R 1 and assume ν(F 0 ) > 0. Define for Lemma 2.8 Assume Y : I × Ω → X is an adapted random periodic path of the Markovian random dynamical system Φ and for any y ∈ X, lim k→∞ Φ(s + kτ, θ(−kτ )ω)y = Y (s, ω) a.s.. Then for any as k → ∞.
Proof: Note for any B ∈ B X , and by the definition of F k and the fact that Thus, by using Lebesgue's dominated convergence theorem, we have that as k → ∞. Then (2.24) follows from Lebesgue's dominated convergence theorem again.

‡ ‡
We now strengthen the above assumption in the following condition to require that the convergence in Lemma 2.8 is exponentially fast, i.e.

Condition (EC):
There exist constants δ 1 , δ 2 > 0 such that for any k ≥ 1 and any B ∈ B X , Under this condition, we call the transition probability P (t, x, ·), t ∈ I + and the periodic measure ρ s ∈ P(X), s ∈ I satisfies Condition (EC). Note no other objects are involved in this condition. Define

Lemma 2.9
Under the condition (EC), we have for any k ≥ 1, Proof: When k ≥ 1, note when kτ ≤ s < (k + 1)τ , Then taking the conditional expectation we have So by condition (EC), We first prove a weak law of large numbers.
Lemma 2.10 (WLLN) Assume Y : I × Ω → X is an adapted random periodic path of the Markovian random dynamical system Φ, and its law and the transition probability of Φ satisfy condition (EC).
Then as N → ∞, Note first that we can prove Eξ k (ω) = 1 ν(F0) F0 ρ s (B)dν(s) by using a similar method as in the proof of (2.21). Now for any m > n, from the measure preserving property of θ and the random periodicity of Y , we have So by Lemma 2.9, |J mn | ≤ δ 1 e δ2 e −δ2(m−n) , when m > n. It is easy to know that when m = n, |J nn | ≤ 1.
On the other hand, Then by Lemma 2.7, we have that in probability as N → ∞. ‡ ‡ Now we can prove the SLLN from the WLLN with the help of Birkhoff's ergodic theorem.
Theorem 2.11 (SLLN) Assume the same conditions as in Lemma 2.10. Then as R ∋ T → ∞, Then by Birkhoff's ergodic theorem, there exists ξ * (ω) such that 26) as N → ∞. On the other hand, by Theorem 2.10, the left hand side of (2.26) converges to That is to say as integer sequence N → ∞, By a standard argument, we can get the result for T → +∞. ‡ ‡ Conversely, we now assume a Markovian random dynamical system has a periodic measure ρ s ∈ P(X). In general, with the original probability space, similar to the case that an invariant measure does not give a stationary process, neither a periodic measure gives a random periodic path. In the following, we will construct an enlarged probability space and an extended random dynamical system, on which the pull-back flow is a random periodic solution. This construction is much more demanding than constructing the periodic measure from a random periodic path. Now we consider a Markovian random dynamical system. If it has a periodic measure on (X, B X ), then we can construct a periodic measure on the product measurable space (Ω × X, F ⊗ B X ). Here we use Crauel's construction of invariant measures on the product space from invariant measures of transition semigroup on phase space. Theorem 2.12 Assume the Markovian random dynamical system Φ has a periodic measure ρ : I → P(X) on (X, B X ). Then for any s ∈ I exists. Let Then µ s is a periodic measure on the product measurable space (Ω ×X, F ⊗B X ) for Φ and E(µ s ) · = ρ s , Proof: First note that if ρ s is a periodic measure on (X, B X ), by Definition 2.5, we have for any This means that ρ 0 is a forward invariant measure under P * (nτ ), n ∈ Z + . By Crauel [10,11], we know that the following limit exists By cocycle property of Φ, we have that for any B ∈ B X for any s ∈ I + , When s ∈ I − , we can also obtain that the above limit still exists by decomposing s = −mτ + s 0 , Now, from the cocycle property and (2.27) and the argument of taking limits in (2.28), we know that for t ∈ I + , It follows that for any A ∈ F ⊗ B X , by (2.11) and (2.29), for t ∈ I + Moreover, It is easy to see that Then µ . is a periodic measure on the product measurable space (Ω × X, F ⊗ B X ) for Φ.
Next let's prove for any B ∈ B X , s ∈ I, E(µ s ) ω (B) = ρ s (B). First, we will show that for any In fact, by the Lebesgue's dominated convergence theorem, the Fubini theorem and measure preserving property of θ, Similarly and also applying the above result, we have for s ∈ I + , In summary, we proved the last claim of the theorem for all s ∈ I.

‡ ‡
We assume that the cocycle Φ generates a periodic probability measure µ on the product measurable space (Ω,F ) = (Ω × X, F ⊗ B). The following observation of an extended probability space, a random dynamical system and the correct construction of an invariant measureμ are key to the proof of the following theorem, which enables us to construct periodic paths from periodic measures. Theorem 2.13 Assume that a random dynamical system Φ generates a periodic probability measure µ on the product measurable space (Ω × X, F ⊗ B X ). Then a measureμ on the measurable space (Ω,F ) defined by,μ is a probability measure andΘ(t) :Ω →Ω defined by (2.30) is measureμ-preserving, and , for any t 1 , t 2 ∈ I + . (2.32) If we extend Φ to a map over the metric dynamical system (Ω,F ,μ, (Θ(t)) t∈I + ) bŷ thenΦ is a RDS on X overΘ and has a random periodic pathŶ : I + ×Ω → X constructed as follows: for anyω * = (s, ω * , x * (ω * )) ∈Ω,

34)
Proof: It is easy to see that the proof of (2.32) is a matter of straightforward computations andμ is a probability measure. To verifyΘ(t)μ =μ, for any t ∈ I + , first using (2.7) and a similar argument as (2.13), we have that for any t ∈ [0, τ ) ∩ I + , It is trivial to note thatΘ(t)μ(∅ × A) =μ(∅ × A). SoΘ(t) isμ-preserving for t ∈ [0, τ ) ∩ I + . This can be easily generalised to any t ∈ I + using the group property ofΘ. Moreover, it is trivial to see thatΦ is a cocycle on X overΘ. Again, the construction ofŶ given by (2.34) is key to the proof, from which the actual proof itself is quite straightforward. In fact, forω = (s, ω, x), we haveŶ (t,ω) = Φ(t+s, θ(−s)ω)x.
Moreover, for any r, t ∈ I + , we have by the cocycle property that Note thatΘ(τ )ω = (s, θ(τ )ω, Φ(τ, ω)x), so we have by the cocycle propertŷ The proof is completed. ‡ ‡ Remark 2.14 It is not clear how to extend the definition of Y to I − in general. However, if the cocycle Φ(t, ω) : X → X is invertible for any t ∈ I + and ω ∈ Ω, for instance in the case of SDEs in a finite dimensional space with some suitable conditions, it is obvious to extend Y to I − .
One implication of Theorems 2.12 and 2.13 is that starting from a periodic measure ρ s ∈ P(X), one can construct a (enlarged) probability space (Ω,F ,μ) and extended random dynamical system, with which the pull-back of the random dynamical system is a random periodic path. In the following we will prove that the transition probability ofΦ(t,ω)x is actually the same as P (t, x, ·) and the law of the random periodic solutionŶ is ρ s , i.e. L(Ŷ (s, ·)) = ρ s , for any s ∈ I, and ρ s+τ = ρ s . We callŶ a random periodic process as its law is periodic. Moreover, we can prove the SLLN of the random periodic processŶ and its associated periodic measure ρ s . This kind of result was previously known for invariant measure and stationary processes as Birkhoff's ergodic theorem (see e.g. Da Prato and Zabczyk [13]). But random periodic processes are more general than stationary processes. Though there are many stationary processes, many processes in the real world are only random periodic, not stationary, e.g. the daily maximum temperature. We believe that the SLLN in this paper opens a new scope of investigating of random periodic processes and their applications in many real world problems. In the following, byÊ we denote the expectation on (Ω,F,μ) Lemma 2.15 Assume ρ s is a periodic measure with respect to the semigroup transition probability of a Markovian random dynamical system Φ. Let the metric dynamical system (Ω,F,μ, (Θ(t)) t∈I + ), the extended random dynamical systemΦ and the random periodic processŶ be defined in Theorem 2.13. Then for any B ∈ B Xμ {ω :Ŷ (t,ω) ∈ B} = ρ t (B), andP (t, y, B) =μ{ω :Φ(t,ω)y ∈ B} = P (t, y, B).
Thus ρ · is a periodic measure with respect toμ as well.

Now definê
We also have: Lemma 2.16 Assume the semigroup transition probability P (t, x, ·) and periodic measure ρ satisfy Condition (EC). Then the random periodic processŶ which is given in Theorem 2.13 has exponentially decay correction in different periods, i.e. for any k ≥ 1, Proof: When k ≥ 1, note when kτ ≤ t < (k + 1)τ , Then taking the conditional expectation and using Lemma 2.15, for k ≥ 1, So by condition (EC), Then we can prove the weak law of large numbers.
Lemma 2.17 (WLLN) Assume the same condition as in Theorem 2.16. Then the random periodic processŶ and its law ρ · satisfy WLLN, i.e. as N → ∞, Proof: Defineξ Then for any m > n, from the measure preserving property of θ and the random periodicity ofŶ , similar to (2.25), we have So by Lemma 2.16, |Ĵ mn | ≤ δ 1 e δ2 e −δ2(m−n) , when m > n. It is easy to know that when m = n, |Ĵ nn | ≤ 1. On the other hand, Thus by Lemma 2.7, we have in probability as N → ∞. ‡ ‡ Now we can prove the SLLN theorem for random periodic processes/periodic measure.
In particular, if Condition (EC) holds for F 0 = [0, τ ), then as T → ∞, Proof: It suffices to note from (2.36) that In particular, if Condition (EC) holds for F 0 = [0, τ ), then as T → ∞, Noteρ F0,T (B) is a random probability measure on (X, B X ) and satisfies For any f ∈ L 1 (X, B X ,ρ F0 (dx)), there is a sequence of simple functions f n such that Therefore for any ǫ > 0, there exists an m * > 0 such that when m ≥ m * , It follows from (2.37) easily that as T → ∞,

Now we can apply Lebesgue's dominated convergence theorem to deduce that as T → ∞,
Thus, there exists T * such that when Here we used the fact thatÊρ F0,N τ =ρ F0 . ThusÊ| X f (x)ρ F0,N τ (dx) − X f (x)ρ F0 (dx)| → 0 as N → ∞. By Chebyshev's inequality we know that X f (x)ρ F0,N τ (dx) → X f (x)ρ F0 (dx) in probability as N → ∞. Therefore there exists a subsequence N k with N k → ∞ as k → ∞ such that On the there hand, using a similar argument of Birkhoff's ergodic theorem as before, we know that converges almost surely. Thus we have as N → ∞,  38)) are equivalent. In other cases considered in this paper, the SLLN with test functions can also be given similarly. We omit them as they become obvious with this remark.

The semi-flow case
As in the last section, denote by (Ω, F , P, (θ(s)) s∈I ) a metric dynamical system and θ(s) : Ω → Ω is assumed to be measurably invertible for all s ∈ I. Denote ∆ := {(t, s) ∈ I 2 , s ≤ t}. Consider a stochastic semi-flow u : ∆ × Ω × X → X, which satisfies the following standard condition u(t, r, ω) = u(t, s, ω) • u(s, r, ω), for all r ≤ s ≤ t, r, s, t ∈ I. (3.1) As in the cocycle case in the last section, we do not assume the map u(t, s, ω) : X → X to be invertible for (t, s) ∈ ∆, ω ∈ Ω. We call u is a τ -periodic stochastic semi-flow if it satisfies an additional periodicity property: there exists a constant τ > 0 such that The definition of random periodic paths (solutions) for stochastic semi-flow was given in [18], [19]: Definition 3.2 A random periodic path of period τ of the semi-flow u : ∆ × Ω × X → X is an F -measurable map Y : I × Ω → X such that for any (t, s) ∈ ∆ and ω ∈ Ω, The following lemma tells how to lift a periodic stochastic semi-flow to a cocycle.
Proof: We use the lifting-up and mapping-down procedure. First, from Lemma 3.3, we know that Y (s, ω) defined in (3.5) is a random periodic solution of cocycleΦ onX. Following the result of Theorem 2.3 about the relation of random periodic solutions and periodic measures for cocycle, there is a periodic measureμ s on the product measurable space (Ω ×X, F ⊗ BX) defined as for any setÃ ∈ F ⊗ BX. Then for any t ∈ I + , s ∈ I, where (μ s ) ω is the factorisation ofμ s (dx, dω) = (μ s ) ω (dx) × P (dω). In fact, (μ s ) ω = δỸ (s,θ(−s)ω) .

‡ ‡
Now consider the case when u(t + s, s, ·) is a Markovian semi-flow on a filtered dynamical system (Ω, F , P, (θ t ) t∈I , (F t s ) s≤t ), i.e. for any s, t, u ∈ I, s ≤ t, we have θ −1 u F t s = F t+u s+u and u(t + s, s, ·) is independent with F s −∞ . We also assume the random periodic solution Y (s, ω) is adapted, that is to say that for any s ∈ I, Y (s, ·) is measurable with respect to F s −∞ := ∨ r≤s F s r . Denote the transition semigroup of u by P (t + s, s, x, B) = P ({ω : u(t + s, s, ω)x ∈ B}), for any B ∈ B X , t ∈ I + , s ∈ I.
Note that u is not a homogenous Markov process, so its transition probability depends on the starting time s. For any probability measure ρ on (X, B X ), define (P * (t + s, s)ρ)(B) = X P (t + s, s, x, B)ρ(dx), for any B ∈ B X , s ∈ I, t ∈ I + . ThenP * (t)ρ s =ρ t+s , andρ s+τ =ρ s , s ∈ I, t ∈ I + . (3.20) Define Then it is easy to see that for any C ∈ B [0,τ )∩I and B ∈ B X , ThenP * (t)ρ =ρ.
Then following Theorem 2.6, we can obtain for any We can take C = [0, τ ) ∩ I to obtain (3.17).

‡ ‡
Similar to the cocycle case, we can also prove a SLLN theorem for the periodic Markovian semi-flow and random periodic paths. We only state the results without including their proofs as they are similar to the cocycle case. First similar with Lemma 2.8, we have: Lemma 3.6 Assume that Y : I × Ω → X is an adapted random periodic path of a periodic Markovian semi-flow u and for any x ∈ X, lim k→∞ u(s, −kτ, ω)x = Y (s, ω) a.s.. Then for any B ∈ B X , as k → ∞.

Condition (ECS):
Assume there exist constants δ 1 , δ 2 > 0 such that for any k ≥ 1, Theorem 3.7 (SLLN) Assume that Y : I × Ω → X is an adapted random periodic path of a periodic Markovian semi-flow u and the law ρ s ∈ P(X) of Y (s) satisfies Condition (ECS). Then as R ∋ T → ∞, In particular, if Condition (ECS) holds for F 0 = [0, τ ), then as T → ∞, In the following, we will give a construction of the random periodic solution from a periodic measure for stochastic semi-flow.

‡ ‡
In the last part of this section, we consider a periodic Markovian semi-flow u. Starting from a periodic measure on (X, B X ) satisfying (3.16), we give a construction of the periodic measure on the product measurable space (Ω × X, F ⊗ B X ) and a therefore random periodic path of the Markovian semiflow u over the enlarged metric dynamical system (Ω,F ,μ, (Θ t ) t∈I + ). Note we ignore the order of ω and s inω = (ω, s, x), then it is easy to see that Θ(t)(ω) = (θ(t)ω,Φ(t, ω)(s, x)). (3.30) In this sense, we can regardΘ as the skew product of the metric dynamical system (Ω, F , P, (θ(s)) s∈I ) and the lifted cocycleΦ : I + ×X →X. The idea is again to lift the semi-flow to the cocycle on the cylinderX and then apply Theorem 2.12 to construct a periodic measureμ s on the product space (Ω ×X, F ⊗ BX ). However, instead of applying Theorem 2.13, which only gives a random periodic path of the lifted cocycleΦ on the cylinderX, here we go further to project the periodic measure to a periodic measure of the semi-flow u on the space (Ω × X, F ⊗ B X ). Then we are ready to construct a random periodic path of the semiflow u on X over the enlarged probability space (Ω,F ,μ, (Θ(t)) t∈I + ) by applying Theorem 3.8.
Theorem 3.9 Assume a periodic Markovian semi-flow u : ∆ × Ω × X → X has a periodic measure (ρ s ) s∈I on (X, B X ) in the sense of (3.16). Then the semi-flow u has a periodic measure µ s , s ∈ I on (Ω × X, F ⊗ B X ) in the sense of (3.11) (or (3.10)) and E(µ s ) · = ρ s .
Proof: As u has a periodic measure ρ s in the sense of (3.16), from the proof of Theorem 3.5, we know that the lifted cocycleΦ onX over the probability space (Ω, F , P ) has a periodic measureρ s defined by (3.22) satisfying (3.20). Then by using Theorem 2.12, there exists a periodic measureμ s , s ∈ I, of Φ on the product measure space (Ω ×X, F ⊗ BX ) over the metric dynamical system (Ω, F , P, (θ s ) s∈I ), i.e. for any s ∈ I, t ∈ I + ,Φ (t, ω)(μ s ) ω = (μ t+s ) θ(t)ω , (μ s+τ ) ω = (μ s ) ω . Moreover, for any C ∈ B [0,τ )∩I and B ∈ B X , from the definition ofΦ, for t ∈ I + , On the other hand, Then it follows from (3.29) that for any s ∈ [0, τ ), t ∈ I + , For general s ∈ I, there is a unique m ∈ Z, s 0 ∈ [0, τ ) such that s = mτ + s 0 . From (3.34) we have It then follows from (3.2) and (3.33) that We can combine Theorem 3.8 and 3.9 to construct a random periodic solutionŶ on the enlarged probability space (Ω,F ,μ, (Θ(t)) t∈I + ), from the periodic measure ρ s for the semi-flow as well. However, unlike in the case of cocycles considered in the last section, the law ofŶ under the probability measurê µ is, though still periodic, but not ρ s any more. Note in the proofL(Ŷ (s)) = ρ s in the cocycle case, the shift invariant plays a key role. This shift invariance can be explained as follows: Let Φ be a cocycle considered in Section 2 and define u(t, s, ω) = Φ(t − s, θ(s)ω), t ≥ s.
Thus, u(t + r, s + r, ω) = u(t, s, θ(r)ω). (3.35) The last identity (3.35) is the shift invariance for cocycles. However, (3.35) is not true for general semi-flows. It is noted that for periodic semi-flow satisfying (3.1) and (3.2), (3.35) is true for r = τ . But this is not enough to prove the result that the law ofŶ is ρ s , which is a key step to prove the law of large numbers for periodic measures. Similarly as in the cocycle case, we have the following SLLN for lifted semi-flows: Theorem 3.10 Assume ρ s is a periodic measure with respect to the Markovian semigroup P (t+ s, s, ·) of the periodic Markovian semi-flow u(t + s, s, ω) on the space X over a metric dynamical system (Ω, F , P, (θ(t)) t∈I ) in the sense of (3.16). Then P (t + s, s, ·) can be lifted to a Markovian semigroup P (t,x, ·) of the Markovian cocycleΦ(t, ·) on the cylinderX = I τ × X, ρ s can be lifted to a τ -periodic measureρ s of the semigroupP (t,x, ·). Furthermore, we can construct an enlarged probability space (Ω,F,μ) and measure preserving mapΘ(t) :Ω →Ω and extended random dynamical systemΦ in the same way as in Theorem 2.13, but with X replaced byX. ThenΦ has a random periodic solution given by: for anyω * = (s, ω * ,x * (ω * )) ∈Ω, Y (t,ω * ) :=Φ(t + s, θ(−s)ω * )x * (θ(−s)ω * ), t ∈ I + .
Assume further that the transition semigroup P (t + s, s, ·) and the periodic measure ρ s satisfy the for any C ∈ B Iτ and B ∈ B X . It is easy to see that they satisfỹ P * (t)ρ s =ρ t+s , andρ s+τ =ρ s , s ∈ I, t ∈ I + .