Empirical Distribution Functions

Let X1, …, Xn be a random sample from a distribution function F on the real line. The empirical distribution function is defined as

It is the natural estimator for the underlying distribution F if this is completely unknown. Because is binomially distributed with mean this estimator is unbiased. By the law of large numbers it is also consistent,

By the central limit theorem it is asymptotically normal,

In this chapter we improve on these results by considering as a random function, rather than as a real-valued estimator for each separately. This is of interest on its own account but also provides a useful starting tool for the asymptotic analysis of other statistics, such as quantiles, rank statistics, or trimmed means.

The Glivenko-Cantelli theorem extends the law of large numbers and gives uniform convergence. The uniform distance is known as the Kolmogorov-Smimov statistic.

19.1 Theorem (Glivenko-Cantelli). If are random variables with distributionfunction F, then.

Proof. By the strong law oflarge numbers, both and for every Given a fixed, there exists a partition. (Points at which F jumps more than e are points of the partition.) Now, for

The convergence of and for every fixed is certainly uniform for in the finite set. Conclude that lim sup, almost surely. This is true for every and hence the limit superior is zero.