Testing for homogeneity in a general one-way classification with fixed effects: power simulations and comparative study

https://doi.org/10.1016/S0167-9473(02)00264-5Get rights and content

Abstract

In an unbalanced one-way layout with fixed effects and heteroscedastic error variances, several tests are available for testing the homogeneity hypothesis of the group means. While it is rather straightforward to study the performance of these tests under the null hypothesis by way of simulation, it is difficult to assess the power of the tests under the alternative hypothesis due to the many possibilities of choosing a particular alternative hypothesis, especially with increasing number of groups. An alternative way of simulating the power of the homogeneity tests is proposed. Only the distance of the alternative hypothesis to the null hypothesis has to be specified.

Introduction

The one-way classification of analysis of variance is one of the most popular models in statistics. In general, interest is in testing the homogeneity of the different group means using the classical ANOVA F-test. However, the standard assumptions of homogeneous error variances and balanced designs are seldomly met in statistical practice, and hence, one has to assume a model with heteroscedastic error variances and unbalanced designs.

The properties of the ANOVA F-test under these non-standard assumptions have been extensively discussed in the literature. It is known that the F-test does not keep its prescribed level, and some alternative procedures for testing the homogeneity hypothesis have been proposed. The finite sample properties of all these tests under the null hypothesis and the alternative hypothesis are now well understood and documented, see for example De Beuckelaer (1996), Mehrotra (1997), Keselman and Wilcox (1999), and Hartung et al. (2002) for recent results. It is straightforward to simulate the actual significance levels of the tests. But, if one is not interested in a particular alternative hypothesis or if one has no idea of choosing a particular alternative hypothesis, it is rather difficult to get an impression of the power of all these tests due to the infinite number of potential alternative hypotheses.

Now, using a reparameterized version of the one-way layout, it is possible to assess the power of the tests over a wide range of alternatives. An algorithm which can easily be implemented is proposed to investigate the power of the tests.

In Section 2, the model and the tests for testing the homogeneity hypothesis are presented and their relationship is discussed. In Section 3, the algorithm is explained, and the results of a simulation study are given in Section 4.

Section snippets

Testing for homogeneity

We consider the one-way classification with fixed effects under the non-standard assumptions of unbalanced designs and heteroscedastic error variances. Let yij be the observation on the jth subject of the ith population or group, i=1,…,K, and j=1,…,ni, K⩾2 and ni⩾2yiji+eij=μ+αi+eij;i=1,…,K,j=1,…,ni,where μ is the common mean for all the K populations, αi is the effect of population i with ∑i=1Kαi=0. The errors eij are assumed to be independent and normally distributedE(eij)=0,Var(eij)=σi2;

Simulation study and algorithm

When a power comparison is made between different competing tests, one has to take the Type I error properties of the tests being compared into account. Hence, we briefly summarize the results concerning the Type I error rate of the tests from previous research. It is known that the ANOVA F-test does not attain acceptable Type I error rates under the presence of heterogeneous error variances, especially when the design is unbalanced, see e.g. De Beuckelaer (1996) and Hartung et al. (2002).

Discussion

Due to the non-standard assumptions (unbalanced designs and heteroscedastic error variances) we make, there does not appear to be any systematic pattern in the simulations, and thus we cannot provide the reader with a general recommendation or a simple take-home message. The boxplots of the Welch and modified Welch test look rather similar, as it was to be expected, since the modified Welch test is a slight modification of the Welch test which has its origins in the Cochran test statistic. The

Acknowledgments

The support of the German Research Community (DFG, Sonderforschungsbereich 475) is gratefully acknowledged. I also thank an Associate Editor and two referees for their careful reading and the useful comments.

References (21)

  • O. Asiribo et al.

    Coping with variance heterogeneity

    Comm. Statist. Theory Methods

    (1990)
  • J.B. Boik

    The Fisher–Pitman permutation testa non-robust alternative to the normal theory F test when variances are heterogeneous

    British J. Math. Statist. Psychol.

    (1987)
  • G.E.P. Box

    Some theorems on quadratic forms applied in the study of analysis of variance problems, I. Effect of inequality of variance in the one-way classification

    Ann. Math. Statist.

    (1954)
  • I.S. Bradbury

    Approximations to permutation distribution in the completely randomized design

    Comm. Statist. Theory Methods

    (1988)
  • M.B. Brown et al.

    The small sample behavior of some statistics which test the equality of several means

    Technometrics

    (1974)
  • W.G. Cochran

    Problems arising in the analysis of a series of similar experiments

    J. Roy. Statist. Soc.

    (1937)
  • A. De Beuckelaer

    A closer examination on some parametric alternatives to the ANOVA F-test

    Statist. Papers

    (1996)
  • B. Efron

    Controversies in the foundations of statistics

    Amer. Math. Monthly

    (1978)
  • J. Gamage et al.

    Size performance of some tests in one-way ANOVA

    Comm. Statist. Simulation Comput.

    (1998)
  • J. Hartung et al.

    Small sample properties of tests on homogeneity in one-way ANOVA and meta-analysis

    Statist. Papers

    (2002)
There are more references available in the full text version of this article.

Cited by (20)

  • Fluorine and metals in Polygonum arenastrum Bor. from areas influenced by various types of industry

    2017, Ecological Indicators
    Citation Excerpt :

    The bioaccumulation factor (BF) was estimated as the ratio of shoot to soil plant available element concentrations with exception of F and Hg where total soil concentrations were used; translocation factor (TF) as the ratio between element concentrations in shoots to roots; concentration factor (CF) as the ratio of root to soil plant available element concentrations with exception of F and Hg where total soil concentrations were used. Shapiro-Wilk’s W test was used for the control of the normality of the analysed features and the Brown-Forsythe test to verify the homogeneity of variances (Argaç, 2004). Box-Cox transformation was used to obtain normal distribution and then one way ANOVA to evaluate differences between mean element concentrations in the P.arenastrum from different sites situated at 0.75, 1.5, 3 and 4.5 km from emission sources (for pooled data per distance) (Zar, 1999).

  • Trace elements in native and transplanted Fontinalis antipyretica and Platyhypnidium riparioides from rivers polluted by uranium mining

    2017, Chemosphere
    Citation Excerpt :

    In measurements and spectral data analysis a personal computer with GENIE software (2000) was used. For verification of the normality of the concentration distribution, Shapiro-Wilk's W test was used, and to check the homogeneity of variances the Brown-Forsythe test was applied (Argaç, 2004). For obtaining normal distribution Box-Cox data transformation was used.

  • δ<sup>34</sup>S values and S concentrations in native and transplanted Pleurozium schreberi in a heavily industrialised area

    2015, Ecotoxicology and Environmental Safety
    Citation Excerpt :

    Differences among the sampling sites in δ34S values and sulphur concentrations in mosses were evaluated by one way ANOVA. The normality of the analysed features was checked using Shapiro–Wilk's W test, and the homogeneity of variances was checked using the Brown–Forsythe test (Brown and Forsythe, 1974, Argaç, 2004). A post-hoc LSD test (Zar, 1999) was used to compare the δ34S values and sulphur concentrations in the moss between the three groups of rural, urban and industrial sites (Sokal and Rohlf, 2003).

  • Mercury in Pleurozium schreberi and Polytrichum commune from areas with various levels of Hg pollution - an accumulation and desorption experiment with microscopic observations

    2014, Ecotoxicology and Environmental Safety
    Citation Excerpt :

    Differences among the sampling sites with respect to the Hg concentrations in the soil and in the moss were evaluated by ANOVA on log-transformed data to obtain a normal distribution of the features according to Zar (1999). The normality of the analyzed features was verified using Shapiro-Wilk׳s W test, and the homogeneity of variances was verified using the Brown–Forsythe test (Brown and Forsythe, 1974; Argaç, 2004). Two-way ANOVAs were used to assess the effects and interactions of the distance from the industry and the type of industry on the concentrations of Hg in the two species.

View all citing articles on Scopus
View full text