A Remark on Unified Error Exponents: Hypothesis Testing, Data Compression and Measure Concentration

Abstract

Let A be finite set equipped with a probability distribution P, and let M be a “mass” function on A. A characterization is given for the most efficient way in which A ⁿ can be covered using spheres of a fixed radius. A covering is a subset C _n of A ⁿ with the property that most of the elements of A ⁿ are within some fixed distance from at least one element of C _n , and “most of the elements” means a set whose probability is exponentially close to one (with respect to the product distribution P ⁿ). An efficient covering is one with small mass M ⁿ(C _n ). With different choices for the geometry on A, this characterization gives various corollaries as special cases, including Marton’s error-exponents theorem in lossy data compression, Hoeffding’s optimal hypothesis testing exponents, and a new sharp converse to some measure concentration inequalities on discrete spaces.