Heteroscedasticity in One Way Multivariate Analysis of Variance

Multivariate Analysis of Variance (MANOVA) can be viewed as a direct extension of the univariate (ANOVA) general linear model that is most appropriate for examining differences between groups of means on several variables simultaneously [1,2]. In ANOVA, differences among various group means on a single-response variable are studied. In MANOVA, the number of response variables is increased to two or more variables. MANOVA has three basic assumptions that are fundamental to the statistical theory: (i) independent, (ii) multivariate normality and (iii) equality of variance-covariance matrices. A statistical test procedure is said to be robust or insensitive if departures from these assumptions do not greatly affect the significance level or power of the test. The violations in assumptions of multivariate normality and homogeneity of covariances may affect the power of the test and type I error rate of multivariate analysis of variance test [3-6].


Introduction
Multivariate Analysis of Variance (MANOVA) can be viewed as a direct extension of the univariate (ANOVA) general linear model that is most appropriate for examining differences between groups of means on several variables simultaneously [1,2]. In ANOVA, differences among various group means on a single-response variable are studied. In MANOVA, the number of response variables is increased to two or more variables. MANOVA has three basic assumptions that are fundamental to the statistical theory: (i) independent, (ii) multivariate normality and (iii) equality of variance-covariance matrices. A statistical test procedure is said to be robust or insensitive if departures from these assumptions do not greatly affect the significance level or power of the test. The violations in assumptions of multivariate normality and homogeneity of covariances may affect the power of the test and type I error rate of multivariate analysis of variance test [3][4][5][6].
The problem of comparing the mean vectors that are more than two multivariate normal populations is called Multivariate Analysis of Variance (MANOVA). If the variance -covariance matrices of the populations are assumed to be equal, then there are some accepted tests available to test the equality of the normal mean vectors, which are: [7] largest root, the trace [8][9][10] likelihood ratio, and the [11,12]. Contrary to popular belief, they are not competing methods, but are complementary to one another. However when the assumption of equality of variance-covariance matrix failed or violated it means that none of the aforementioned test statistic is appropriate for the analysis otherwise the result will be prejudiced. This predicament is known as the multivariate Behrens -Fisher problem which deal with testing the equality of normal mean vector under heteroscedasticity of dispersion matrices. If the covariance matrices are unknown and arbitrary, then the problem of testing equality of the mean vectors is more complex, and only approximate solutions are available.
Johansen et al. [13][14][15] proposed multivariate tests for the situation in which the covariance matrices could be unequal. In this study, an approximate degree of freedom used [16] for comparing k normal mean vectors when the population variance -covariance matrices are unknown is proposed and compared with an existing procedure (by Johanson) when the groups (k) and random variables (p) are three respectively.

Methodology
Let x ij … x in be a sample from a p-variate normal distribution with mean vector µ i and covariance matrix ∑i=1,... k, assuming that all the samples are independent. Let sample mean and sample covariance matrix be i x and S i respectively based on the i th sample.
Where W p (r, ∑) denotes the p-dimensional Wishart distribution with degrees of freedom (df=r) and scale parameter matrix ∑.
The problem of interest here is to test: The first two central moments of L are obtained: We shall consider the test statistic 1 ' y S y − and use Univariate Satterthwaite approximation of degrees of freedom method to suggest multivariate generalization based on the T 2 -distribution [22][23][24][25][26][27][28][29]. Let Eqn. (12)  ....

Proposed Method
The entire aforementioned scholars worked on the degree of freedom by using various methods to get approximate degree of freedom to the test statistic, which the proposed procedure intended to, by extending Satterthwaite's procedure (two moment solution to the behrens-fisher problem) in univariate to a multivariate Behrens-Fisher problem. In Satterthwaite [16] proposed a method to estimate the distribution of a linear combination of independent chi-square random variables with a chi-square distribution. Let Since linear combination of random variable does not, in general, possess a chi-square distribution. Satterthwaite [16] suggested the use of a chi-square distribution, Say ( Where "~"  is taken to mean "is approximately distributed as." From an intuitive standpoint, the distribution of [ ] should have characteristics similar to some member of the chi-square family of densities [17][18][19][20][21]. But recall that if a chi-square distribution has degrees of freedom (n i -1), then its mean is and variance is 2(n i -1).
Symbolically, this requires that, the first moment of the statistic is: This implies that a chi-square with f degrees of freedom should be used.
Let consider the second moment. The variance of the statistic is:

Data simulation
Data was simulated in R environment to estimate power of the test and Type I error rate when the alternative hypothesis is true (that is when the mean vectors are not equal).

Data analysis
Simulated and real life data sets from previous study [17] were used to compare the proposed/alternative procedure with the existing one (Johanson). For the simulated data, three factors were varied namely; number of groups (k), the number of variables (p) and significant levels (α).
In each of the 1000 replications and for each of the factor combination, an i n p × (where i=1,….4) data matrix X i were generated using an R package for Multivariate Normal. The programme also performs the Box-M test for equality of covariance matrices using the test statistic: ( ) H o is rejected at the significance level α if Table 1 shows that irrespective of the sample size and significant level α, the propose procedure has the higher power of the test and less type I error rate compared to Johanson when the alternative hypothesis is true. The two only have the same type I error rate when the sample sizes are large (100's and 200's), but then the powers of the test are not the same throughout the sample sizes considered (5's, 10's, 50's, 100's and 200's).

Result
From Table 2, when the sample size are not equal and very small [ (5,10,15) and (20,25,30)], Johanson procedure perceived to be better than the propose procedure in terms of power of the test but poor in type I error rate at significant level α=0.01, but when sample sizes increases to (50,70,90) and (100,150,200) the propose procedure performed better at the two significant level (α=0.01 and 0.05). Table 3, when the sample sizes are small [ (5,5,5) and (10,10,10)] and equal in all the groups, Johanson performed better at significant level α=0.01 in terms of power of the test while propose procedure are better in terms of type I error rate in all the scenario, but when sample sizes are (100,100,100) and (200,200,200) they both perform the same.
From Table 4, when the stimulated data are multivariate gamma and unbalance, the propose procedure are better than Johanson procedure in the entire scenario both in terms of power of the test and type I error rate.

Illustrative example
The real life data used by Krishnamoorthy and Xia [27] was used  to compare propose procedure with Johanson procedure so as to understand the behavior of these two tests as described early on, for comparing several groups. There are five samples of 30 skulls from each of the early predynastic period (circa 4000 BC), the late predynastic period (circa 200 BC), and the Roman period (circa AD 150). Four measurements are available on each skull, namely. X 1 =maximum breadth, X 2 =borborygmatic height, X 3 =dentoalveolar length, and X 4 =nasal height (all in mm). And n i =...=n 4 =15, the number of groups is k=4, while the number of variables is p=4. The null hypothesis of interest is whether the mean vectors for the four variables are the same across the four periods [31,32]. The hypothesis may be written as 3 Table 5 above, when significant level α is 0.05 Johanson and propose procedure rejected null hypothesis because test statistic is greater than critical value that is 2.275 is greater than 2.0443 and 7.6763 is greater than 2.6138, also p-values of Johanson is 0.0294 which is less than 0.05 and that of propose procedure is 0.0001 which is less than 0.05, but when significant level α are 0.025 and 0.01, Johanson accepted the null hypothesis because 2.275 is less than 2.3451 and 2.7464 with p-value greater than α, while propose procedure rejected null hypothesis since 7.6763 is greater than 3.1377 and 3.8459 with p-value less than α.

Remark
From the simulated data, it is obvious that the propose procedure performed better than Johanson procedure because its (propose procedure) power of the test are higher than that of Johanson procedure in the entire scenario that is, when sample size differs, when significant level α varies, when the design are balance and unbalance. Also from the illustrative example, it is observed that propose procedure performed than Johanson procedure because propose procedure has the higher power of the test than Johanson.