Probability and Moment Approximations Using Limit Theorems

Abstract

So far we have described the methods for determining the exact probability of events using probability mass functions (PMFs) for discrete random variables and probability density functions (PDFs) for continuous random variables. Also of importance were the methods to determine the moments of these random variables. The procedures employed were all based on knowledge of the PMF/PDF and the implementation of its summation/integration. In many practical situations the PMF/PDF may be unknown or the summation/integration may not be easily carried out. It would be of great utility, therefore, to be able to approximate the desired quantities using much simpler methods. For random variables that are the sum of a large number of independent and identically distributed random variables this can be done. In this chapter we focus our discussions on two very powerful theorems in probability–the law of large numbers and the central limit theorem. The first theorem asserts that the sample mean random variable, which is the average of lID random variables and which was introduced in Chapter 14, converges to the expected value, a number, of each random variable in the average. The law of large numbers is also known colloquially as the law of averages. Another reason for its importance is that it provides a justification for the relative frequency interpretation of probability. The second theorem asserts that a properly normalized sum of lID random variables converges to a Gaussian random variable.