Improving power of multivariate combination-based permutation tests

Abstract

Developing powerful hypothesis testing procedures devoted at comparing multivariate populations is quite a common and relevant topic either from the methodological and the practical point of view and in this connection the NonParametric Combination (NPC) permutation methodology provides a more flexible and effective background for many multivariate testing problems (Pesarin and Salmaso in Permutation tests for complex data: theory, applications and software, 2010a). The goal of this paper is to propose some specific procedures aimed at possibly improving power of NPC Tests in the context of the additive linear model. It will be shown by an extensive simulation study, the improved-in-power NPC Tests are certainly good alternatives with respect to the traditional multivariate tests such as Hotelling T ² and multivariate rank-based tests, especially in cases of heavy-tailed distributions. Moreover, the NPC methodology offers several advantages since it provides robust solutions with respect to the true underlying random error distribution and it is not affected by the problem of the loss of degrees of freedom when keeping fixed the number of observations. Indeed, unlike traditional methods, when the number of informative variables increases its power monotonically increases as well (leading to the so-called finite-sample consistency property of NPC Test, Pesarin and Salmaso in J. Nonparametr. Stat. 22(5):669–684, 2010b).

Appendix: Extension of finite-sample consistency to non-associative statistics

In order to extend finite-sample consistency to non-associative statistics, let us briefly introduce the notion of conditional (permutation) unbiasedness for any kind of statistics $T^{*} (\boldsymbol{\delta})=S(\mathbf{Y}_{1}^{*}(\boldsymbol{\delta}))- S(\mathbf{Y}_{2}^{*})$. To this end and with clear meaning of the symbols, let us observe that:

• T ^o(0)=S(Z ₁)−S(Z ₂), i.e. the null observed value of statistic T.

• T ^o(δ)=S(Z ₁+δ)−S(Z ₂)=S(Z ₁)+D _S (Z ₁,δ)−S(Z ₂)=T ^o(0)+D _S (Z ₁,δ), where D _S (Z ₁,δ)≥0.

• $T^{*} (0)=S(\mathbf{Z}_{1}^{*} )- S(\mathbf{Z}_{2}^{*})$, i.e. the value of T in the permutation $\mathbf{u}^{*} = u_{1}^{*},\dots,u_{n}^{*}$.

• $T^{*} (\boldsymbol{\delta})=S(\mathbf{Z}_{1}^{*} +\boldsymbol{\delta}^{*} )- S(\mathbf{Z}_{2}^{*} )=T^{*} (0)+D_{S}(\mathbf{Z}_{1}^{*},\boldsymbol{\delta}^{*} )- D_{S}(\mathbf{Z}_{2}^{*})$.

• $D_{S}(\mathbf{Z}_{1}^{*},\boldsymbol{\delta}^{*} )\geq D_{S}(\mathbf{Z}_{2}^{*},0)=0=D_{S}(\mathbf{Z}_{2},0)$, because effects $\boldsymbol{\delta}_{2i}^{*}$ coming from first group are non-negative.

• $D_{S}(\mathbf{Z}_{1}^{*},\boldsymbol{\delta}^{*} )\leq D_{S}(\mathbf{Z}_{1}^{*},\boldsymbol{\delta})$ point-wise, because in $D_{S}(\mathbf{Z}_{1}^{*},\boldsymbol{\delta}^{*})$ there are non-negative effects assigned to units coming from group 2; e.g., suppose n ₁=3, n ₂=3, and u ^∗=(3,5,4,1,2,6), then $(\mathbf{Z}_{1}^{*},\boldsymbol{\delta}^{*} )=[(Z_{13},\boldsymbol{\delta}_{13}), (Z_{22},0),(Z_{21},0)]$, and so

  $$\bigl(\mathbf{Z}_{1}^{*},\boldsymbol{\delta}\bigr)= \bigl[(Z_{13},\boldsymbol{\delta}_{13}),(Z_{22}, \boldsymbol{\delta}_{11}),(Z_{21},\boldsymbol{ \delta}_{12})\bigr], $$

  or

  $$\bigl(\mathbf{Z}_{1}^{*},\boldsymbol{\delta}_{1} \bigr)=\bigl[(Z_{13},\boldsymbol{\delta}_{13}),(Z_{22}, \boldsymbol{\delta}_{12}),(Z_{21},\boldsymbol{ \delta}_{11})\bigr]; $$

  it is to be emphasized that $Y(u_{i}^{*})=Z(u_{i}^{*})+\boldsymbol{\delta}(u_{i}^{*})$ if $u_{i}^{*} \leq n_{1}$, that is units coming from first group maintain their effects, whereas the rest of effects are randomly assigned to units coming from second group.

• $D_{S}(\mathbf{Z}_{1}^{*},\boldsymbol{\delta})\mathop{ =} \limits^{d} D_{S}(\mathbf{Z}_{1},\boldsymbol{\delta})$, because $\mathrm{Pr}\{\mathbf{Z}_{1}^{*} |\mathsf{X}_{/\mathbf{Y}(0)} \}= \mathrm{Pr}\{\mathbf{Z}_{1} |\mathsf{X}_{/\mathbf{Y}(0)}\}$ (see Pesarin and Salmaso 2010a).

Thus $D_{S}(\mathbf{Z}_{1}^{*},\boldsymbol{\delta}^{*} )- D_{S}(\mathbf{Z}_{2}^{*} )\leq D_{S}(\mathbf{Z}_{1},\boldsymbol{\delta})$ in permutation distribution and so

$$\begin{aligned} \lambda_{T} \bigl(\mathbf{X}(\boldsymbol{\delta})\bigr) & = \Pr \bigl\{ T\bigl(\mathbf{X}^{*} (\boldsymbol{\delta})\bigr)\geq T\bigl( \mathbf{X}(\boldsymbol{\delta})\bigr)|\mathsf{X}_{/\mathbf{Y}(\boldsymbol{\delta})}\bigr\} \\ & = \Pr \bigl\{ T^{*} (0)+D_{S}\bigl( \mathbf{Z}_{1}^{*},\boldsymbol{\delta}^{*} \bigr)- D_{S}\bigl(\mathbf{Z}_{2}^{*} \bigr)\\ &\quad {}- D_{S}(\mathbf{Z}_{1},\boldsymbol{\delta})\geq T^{o} (0)|\mathsf{X}_{/\mathbf{Y}(0)}\bigr\} \\ & \leq \Pr \bigl\{ T^{*} (0)\geq T^{o} (0)| \mathsf{X}_{/\mathbf{Y}(0)} \bigr\} = \lambda_{T}\bigl(\mathbf{Y}(0)\bigr), \end{aligned}$$

which establishes the dominance in permutation distribution of λ _T (Y(δ)) with respect to λ _T (Y(0)), uniformly for all data sets Y 2Y ⁿ, for all underlying distributions P, and for all associative and non-associative statistics T=S(Y ₁)−S(Y ₂).

These results allows us to prove the following:

Theorem

Suppose that in a two-sample problem there are p≥1 non homoscedasticity variables Y=(Y ₁,…,Y _p ), the observed data set is Y(δ)=(δ _k +σ _k Z _i1k ,i=1,…,n ₁,σ _k Z _i2k ,i=1,…,n _2; k=1,…,p) and the hypotheses are

$$\begin{aligned} & H_{0}: \Bigl[ \mathbf{Y}_{1}\mathop{ =} \limits ^{d} \mathbf{Y}_{2} \Bigr] = [ \boldsymbol{\delta} = \mathbf{0} ]\quad \mathrm{against}\\ & H_{1}: \Bigl[ \textbf{Y}_{1}\mathop{ >} \limits^{d} \textbf{Y}_{2} \Bigr] = [ \boldsymbol{\delta} > \mathbf{0} ], \end{aligned}$$

where δ=(δ ₁,…,δ _p )′. For the testing purpose consider the statistic

$$T_{k}^{*}(\boldsymbol{\delta})=1/p \sum _{ k \leq p}\bigl[\tilde{Y}_{1k}^{*}( \delta_{k})- \tilde{Y}_{2k}^{*} \bigr]/S_{k}, $$

where $\tilde{Y}_{ji}^{*}(\boldsymbol{\delta})=\mathsf{M}d[Y_{ijk}^{*}(\boldsymbol{\delta})/S_{Mk}, k=1,\dots,p]$, i=1,…,n _j , j=1,2, is the median vector of p scale-free variables specific to i-th subject, and $S_{Mk} = \mathit{MAD}_{k} =\mathsf{M}d[|Y_{\mathit{ijk}} - \tilde{Y}_{k}|, i=1,\dots,n_{j}, j=1,2]$ is the median of absolute deviations from the median specific to the variable Y _k .

In this setting, the test based on $T_{Md}^{*} (\delta)$ is conditional and unconditional finite-sample consistent as far as p diverges and M d(Y ₁(δ))>0 without requiring existence of any positive moment for p variables.

Proof

For the non-associative statistics it applies the uniformly stochastic ordering of the significance level functions with respect to δ and Y, that is for δ′>δ

$$\mathrm{Pr}\bigl\{ \lambda_{T} [\mathbf{Y}( \boldsymbol{\delta} ')]\leq \alpha \bigr\} \mathop{ \le} \limits^{d} \mathrm{Pr}\bigl\{ \lambda_{T} [\mathbf{Y}( \boldsymbol{\delta} )]\leq \alpha\bigr\} , $$

hence, with reference to the finite-sample consistency of the second order combined test using the medians

$$\begin{aligned} T''^{obs} & = 1/p \sum _{ k \leq p}\bigl[\tilde{Y}_{1k}( \delta_{k})- \tilde{Y}_{2k}(0)\bigr]/S_{k}\\ & = 1/p \sum _{ k \leq p}\bigl[\tilde{Y}_{1k}(0)+ \delta_{k} - \tilde{Y}_{2k}^{*}(0)\bigr]/S_{k}\\ & = 1/p \sum_{ k \leq p} \bigl[\tilde{Y}_{1k}(0)- \tilde{Y}_{2k}(0)\bigr]/S_{k} +1/p \sum _{ k \leq p} \delta_{k} /S_{k}. \end{aligned}$$

It should be noted that the quantity $1/p \sum_{ k \leq p} [\tilde{Y}_{1k}(0)- \tilde{Y}_{2k}(0)]/S_{k}$ is nothing else than the arithmetic mean of p sample differences which are all measurable, given that all p involved variables are non-degenerate by assumption (i.e. S _k >0;k=1,…,p) and, provided that min(n ₁,n ₂) is not too small, are all finite (for instance, with the Pareto distribution if its parameter is γ≥[min(n ₁,n ₂)/2]; where [⋅] is the integer part of (⋅), the first moment E _Y (Y,γ) is finite; it is noticeable that E _Y (Y,γ) does not exist γ≤1). Thus, by the law of large numbers for sequences of dependent variables, as p diverges it converges weakly to a constant, not necessarily null. The induced standardized global noncentrality 1/p∑ _k≤p δ _k /S _k , which is itself a mean of non-negative and measurable quantities, if it converges, it does so to a positive quantity but it could be let free to diverge as well. □