Properties of Shannon and Rényi entropies of the Poisson distribution as the functions of intensity parameter

We consider two types of entropy, namely, Shannon and Rényi entropies of the Poisson distribution, and establish their properties as the functions of intensity parameter. More precisely, we prove that both entropies increase with intensity. While for Shannon entropy the proof is comparatively simple, for Rényi entropy, which depends on additional parameter α > 0 , we can characterize it as nontrivial. The proof is based on application of Karamata’s inequality to the terms of Poisson distribution.


Introduction
The concept and formulas for different types of entropy, which come mainly from information theory, are now widely used, in many applications, in par-is proved with the help of Karamata's inequality.Some by-product inequalities are obtained in Section 4, while auxiliary statements are postponed to Appendix.

Analytical properties of Shannon entropy of
Poisson distribution as the function of intensity parameter λ Consider a discrete distribution {p i , i ≥ 1}.Its Shannon entropy is defined as In particular, we consider Poisson distribution with parameter λ: Its Shannon entropy H S (λ) equals It is natural to assume that Shannon entropy H S (λ) of Poisson distribution strictly increases with λ ∈ (0, +∞).We will prove this result, as well as the concavity property of H S (λ), in the next statement.
Proof.Obviously, we can differentiate the series (1) term by term for any λ > 0, and get the equality It is clear that both terms in the right-hand side of (2) are non-negative for for λ ∈ (0, 1], and the second one is strictly positive, therefore So, H ′′ S (λ) < 0 for all λ > 0. Therefore, H ′ S (λ) strictly decreases in λ and it is sufficient to prove that lim λ→∞ H ′ S (λ) ≥ 0. However, and it is sufficient to establish that This inequality is proved in Lemma A.1.Finally, we get that H ′ S (λ) > 0 for all λ ≥ 0 and H ′′ S (λ) < 0 for all λ ≥ 0, whence the proof follows.
3 Analytical properties of Rényi entropy of Poisson distribution as the function of intensity parameter λ Again, consider a discrete distribution {p i , i ≥ 1}.Its Rényi entropy is defined as and , where H S (p i , i ≥ 1) is a Shannon entropy.In the case of Poisson distribution As for the Shannon entropy, our goal is to investigate the behaviour of Rényi entropy of Poisson distribution as the function of intensity λ.To be more precise, we wish to prove that for any fixed α > 0, α ̸ = 1, Rényi entropy of Poisson distribution increases in λ.Let us take into account equality (3) and consider two cases.
To prove this result, we will apply Karamata's inequality (see Lemma A.2 in Appendix).This inequality deals with non-increasing sequences, so it suggests to rearrange the terms of the Poisson distribution in non-increasing order and study the properties of the resulting sequence.
Firstly, we will establish that for every λ > 0 the number S n (λ) is equal to the sum of some n + 1 consecutive terms of the initial sequence {p k (λ) :

By-product inequalities
As can be seen from Section 3, the proof of increasing (decreasing) properties of function ψ(α, λ) did not use differentiation of ψ(α, λ) and the properties of its derivatives.However, function ψ(α, λ) can be differentiated term by term at any point α > 0 in λ > 0 and as a result, we shall get after some elementary transformations that for any α, λ > 0 Obviously, if α = 1 then for any λ > 0 the derivative is zero because it differs by a strictly positive multiplier αe −αλ from the expression However, taking into account Theorem 2, we can establish some nontrivial inequalities.
Remark 1.The nontrivial character of inequalities ( 8) and ( 9) is implied by the fact that we can not compare the respective series term by term, and the relation between kth terms depends on whether the condition λ > k or the inverse one is satisfied.The similar situation was with ψ(α, λ), but now, having Theorem 2 in hand, we do not need to analyze the series in more detail in order to compare them.
Respectively, we see from Fig. 7 that for λ > 0 and α > 0 (the values of α are chosen from 1.1 to 2.0 with interval 0.1) R(α, λ) < 0. The same is confirmed by the surface on Fig.A Appendix Lemma 3. The following relation holds: Proof.It is sufficient to consider λ > 4. Let us rewrite the sum under the sign of a limit as follows: where λ 2 is the floor function of λ 2 (i.e. the greater integer less or equal to λ 2 ).Obviously, .

Now
So, we can bound e −λ (log λ) −1 S 1 (λ) as λ → ∞, as follows: . Now shift the range of the values of λ under consideration to λ > 42 and note that for such λ we have the bounds λ Obviously, e λ = ∞ k=0 λ k k! , while it follows immediately, that similarly to previous calculations in (10) 0 ≤ lim inf Relation (12) implies that the following limit exists: The proof immediately follows from (10), (11) and (13).
λ) is an increasing function of λ when α is fixed,
, according to two-sided bounds of factorial, for any n > 1