Level Sets Semimetrics for Probability Measures with Applications in Hypothesis Testing

Abstract

In this paper we introduce a novel family of level sets semimetrics for density functions and address subtleties entailed in the estimation and computation of such semimetrics. Given data drawn from f and q, two unknown density functions, we consider different level set semimetrics so to test the null hypothesis $H_0: f=q$. The performance of such testing procedure is showcased in a Monte Carlo simulation study. Using the methods developed in the paper, we assess differences in gene expression profiles between two groups of patients with different respiratory recovery patterns in a clinical study; and find significant differences between the 15 top–ranked genes density profiles corresponding to the two groups.

Technical Appendix

The semimetric in Equation (1) is non-negative and symmetric by definition; and also obeys the triangular inequality (see the Biotope transformation in Deza and Deza (2009), pp 118 for further details).

Proposition: The semimetric $\mathrm {D}(f,q,\varvec{\nu },\mathbf {w})=\sum _{i=1}^{k}w_i \mathrm {d}\left( \mathcal {A}_i(f,\varvec{\nu }),\mathcal {A}_i(q,\varvec{\nu })\right)$ behaves as a proper metric when: (1) $\mathrm {d}:\mathcal {X}\times \mathcal {X} \rightarrow \mathbb {R}^+$ is a metric between subsets in $\mathcal {X}$, and (2) $\varvec{\nu }_k\equiv \{\nu _0,\dots ,\nu _k\}$ is an asymptotically dense set in [0, 1]. Moreover, in the limit $\mathrm {D}(f,q,\varvec{\nu }_k,{ {w}})$ does not depend on the sequence $\varvec{\nu }_k$ (only depends of ${ {w}}$).

Proof: We need to show that $\mathrm {D}(f,q,\varvec{\nu }_k,{ {w}}){\mathop {\longrightarrow }\limits ^{ n\rightarrow \infty }}0$ if and only if $f=q$. Consider the asymptotically dense set $\varvec{\nu }_k = \{ \frac{i}{k}\}_{i=0}^k$, then if $f = q$ for all $k\in \mathbb {N}$ it holds that $d_i(f,q,\varvec{\nu }_k)=0$ for $i \in \{1,\dots ,k\}$, leading to $\mathrm {D}(f,q,\varvec{\nu }_k,{ {w}})=0$. When $f \ne q$, there exists a constant $k_0$ such that for all $k > k_0$, $d_i(f,q,\varvec{\nu }_k) > 0$ for at least one $i \in \{1,\dots ,k\}$; therefore for $k>k_0: \mathrm {D}(f,q,\varvec{\nu }_k,{ {w}})>0$ and then $\lim \limits _{k \rightarrow \infty } \mathrm {D}_J(f,q,\varvec{\nu }_k,{ {w}})>0$. Notice also that for any asymptotically dense set $\varvec{\nu }_k$ then $\lim \limits _{k \rightarrow \infty } \mathrm {D}_J(f,q,\varvec{\nu }_k,{ {w}})=D(f,q,{ {w}})$; since for all asymptotically dense sequences $\varvec{\nu }_k$ we obtain (asymptotically) the same collection of level sets to compute $D(f,q,{ {w}})$.