Abstract
Motivated by the genomic application of expression quantitative trait loci (eQTL) mapping, we propose a new procedure to perform simultaneous testing of multiple hypotheses using Bayes factors as input test statistics. One of the most significant features of this method is its robustness in controlling the targeted false discovery rate even under misspecifications of parametric alternative models. Moreover, the proposed procedure is highly computationally efficient, which is ideal for treating both complex system and big data in genomic applications. We discuss the theoretical properties of the new procedure and demonstrate its power and computational efficiency in applications of single-tissue and multi-tissue eQTL mapping.
Similar content being viewed by others
References
Barreiro LB, Tailleux L, Pai AA, Gicquel B, Marioni JC, Gilad Y (2012) Deciphering the genetic architecture of variation in the immune response to Mycobacterium tuberculosis infection. Proc Natl Acad Sci USA 109(4):1204–1209
Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B 57(1):289–300
Cox DR, Hinkley D (1979) Theoretical statistics. Chapman & Hall, London
De la Cruz O, Wen X, Ke B, Song M, Nicolae DL (2010) Gene, region and pathway level analyses in whole-genome studies. Genet Epidemiol 34(3):222–231
DiCiccio TJ, Kass RE, Raftery A, Wasserman L (1997) Computing Bayes factors by combining simulation and asymptotic approximations. J Am Stat Assoc 92(439):903–915
Dimas AS, Deutsch S, Stranger BE, Montgomery SB et al (2009) Common regulatory variation impacts gene expression in a cell type-dependent manner. Science 325(5945):1246–1250
Efron B, Tibshirani R, Storey JD, Tusher V (2001) Empirical bayes analysis of a microarray experiment. J Am Stat Assoc 96(456):1151–1160
Flutre T, Wen X, Pritchard J, Stephens M (2013) A statistical framework for joint eQTL analysis in multiple tissues. PLoS Genet 9(5):e1003486
Genovese C, Wasserman L (2004) A stochastic process approach to false discovery control. Ann Stat 32(3):1035–1061
Good I (1992) The bayes/non-bayes compromise: a brief review. J Am Stat Assoc 87(419):597–606
Ji Y, Lu Y, Mills GB (2008) Bayesian models based on test statistics for multiple hypothesis testing problems. Bioinformatics 24(7):943–949
Johnson VE (2005) Bayes factors based on test statistics. J R Stat Soc Ser B 67(5):689–701
Johnson VE (2008) Properties of Bayes factors based on test statistics. Scand J Stat 35(2):354–368
Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90(430):773–795
Liang F, Paulo R, Molina G, Clyde MA, Berger JO (2008) Mixtures of g priors for Bayesian variable selection. J Am Stat Assoc 103(481):410–423
Müller P, Parmigiani G, Robert C, Rousseau J (2004) Optimal sample size for multiple testing: the case of gene expression microarrays. J Am Stat Assoc 99(468):990–1001
Müller P, Parmigiani G, Rice K (2006) FDR and Bayesian multiple comparisons rules. In: Bayesian statistics 8, vol 0. Oxford University Press, p 349–370
Newton MA, Noueiry A, Sarkar D, Ahlquist P (2004) Detecting differential gene expression with a semiparametric hierarchical mixture method. Biostatistics 5(2):155–76
Opgen-Rhein R, Strimmer K (2007) Accurate ranking of differentially expressed genes by a distribution-free shrinkage approach. Stat Appl Genet Mol Biol 6
Raftery AE (1996) Approximate Bayes factors and accounting for model uncertainty in generalised linear models. Biometrika 83(2):251–266
Saville BR, Herring AH (2009) Testing random effects in the linear mixed model using approximate bayes factors. Biometrics 65(2):369–376
Servin B, Stephens M (2007) Imputation-based analysis of association studies: candidate regions and quantitative traits. PLoS Genet 3(7):e114
Storey JD (2002) A direct approach to false discovery rates. J R Stat Soc Ser B 64(3):479–498
Storey JD (2003) The positive false discovery rate: a Bayesian interpretation and the q-value. Ann Stat 31(6):2013–2035
Storey JD (2007) The optimal discovery procedure: a new approach to simultaneous significance testing. J R Stat Soc Ser B 69(3):347–368
Storey JD, Tibshirani R (2003) Statistical significance for genomewide studies. Proc Natl Acad Sci USA 100(16):9440–9445
Storey JD, Taylor JE, Siegmund D (2004) Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach. J R Stat Soc Ser B 66(1):187–205
Sun W, Cai TT (2007) Oracle and adaptive compound decision rules for false discovery rate control. J Am Stat Assoc 102(479):901–912
Wakefield J (2009) Bayes factors for genome-wide association studies: comparison with P-values. Genet Epidemiol 33(1):79–86
Wen X (2014) Bayesian model selection in complex linear systems, as illustrated in genetic association studies. Biometrics 70(1):73–83
Wen X, Stephens M (2014) Bayesian methods for genetic association analysis with heterogeneous subgroups: from meta-analyses to gene-environment interactions. Ann Appl Stat 8(1):176–203
Whittemore AS (2007) A Bayesian false discovery rate for multiple testing. J Appl Stat 34(1):1–9
Acknowledgments
We thank Debashis Ghosh, Matthew Stephens, and Timothee Flutre for their fruitful discussion and feedbacks. We are grateful for the insightful comments from the two anonymous reviewers. This work is supported by the NIH Grant R01 MH101825 (PI: M.Stephens).
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Proof of Proposition 1
Proof
Suppose that the true generative distributions of \({{\varvec{Y}}}_i\) under the null and alternative model are given by \(p^{0^*}_i\) and \(p^{1^*}_i\), respectively. As sample size \(n_i\) is sufficiently large for each test, the true Bayes factor, \(\mathrm{BF}_i^*\), has the following properties
Assumption 1, in contrast, only requires that the assumed Bayes factors satisfy
Under the conditions stated in Assumption 2, \(\Pr \left( {\hat{\pi }}_0 \ge \pi _0 \mid \pi _0 \right) \rightarrow 1\) implies that \(\Pr \left( {\hat{\pi }}_0 \ge \pi _0 \mid {{\varvec{\mathcal {Y}}}}\right) \rightarrow 1\) for an arbitrary prior distribution on \(\pi _0\). Let \(\epsilon := \Pr \left( {\hat{\pi }}_0 < \pi _0 \mid {{\varvec{\mathcal {Y}}}}\right) \). Then, it follows that
Importantly, the large number of the null tests is required for ensuring the upper-bound property of \({\hat{\pi }}_0\).
Consequently,
By (6.1), as \(n_i \rightarrow \infty \),
whereas by (6.2),
Hence,
and by (6.3) and (6.4), each individual test satisfies
The decision rule stated in the Proposition 1 yields a true Bayesian FDR
By (6.8), it is clear that
Therefore, the Bayesian FDR can be consistently controlled. Furthermore, controlling \(\frac{\sum _{i=1}^m \delta ^\dagger _i (1-{\hat{v}}_i)}{D^\dagger \vee 1} \le \alpha \) ensures that
because \(\overline{\text {FDR}}\) is obviously bounded. \(\square \)
Appendix 2: Proof of Lemma 2
Proof
Let \((\mathrm{BF}_{(1)},\dots ,\mathrm{BF}_{(m)})\) denote the order statistics from the m Bayes factors that are all generated from the respective null hypotheses. Let \(M_j := \frac{1}{j} \sum _{i=1}^j \mathrm{BF}_{(i)}\) denote the partial sample mean computed by the EBF procedure. Note that the sequence \(M_1, M_2, \dots \) is monotonically non-decreasing. Furthermore, by the law of large numbers and the result of Lemma 1, it follows that
for sufficiently large m.
In the case that \(d_0 < m\), it must be true that
Because the truncated sample mean from the order statistics \((\mathrm{BF}_{(d_0+1)}, ... , \mathrm{BF}_{(m)})\) converges to a quantity that is strictly >1, their contribution to the overall sample mean, \(M_m\), must be negligible, i.e.,
Taken together, we have shown that
\(\square \)
Appendix 3: Proof of Proposition 2
Proof
In the general mixture case, let \(\mathcal {S}^0\) denote the subset of Bayes factors whose data are generated from the null models. Based on Lemma 2, applying the EBF procedure on \(\mathcal {S}^0\) results in an estimate
where \(|\mathcal {S}^0|\) denotes the cardinality of \(\mathcal {S}^0\). In the mixed samples, (6.14) suggests that
The LHS should be strictly <1 if there exists small values of Bayes factors from the alternative models.
Because the EBF procedure finds the largest subset whose sample mean is <1, it must hold true that
and thus we conclude that
\(\square \)
Appendix 4: Proof of Corollary 1
Proof
Given a predefined FDR level \(\alpha \), the rejection threshold on the estimated false discovery probabilities is given by
Equivalently when there is at least one rejection, the above rejection threshold implies that the rejection set \(\Omega := \{i: {\hat{v}}_i > t^\dagger _{\alpha }\}\) is the largest set such that
where \(\Pr (Z_i = 0 \mid {{\varvec{Y}}}_i, {\hat{\pi }}_0) = (1-{\hat{v}}_i)\) and \(|| \Omega || = D^\dagger (t_{\alpha }^\dagger )\) denotes the cardinality of the set \(\Omega \). That is, the average estimated false rejection probability in the rejection set should be \(\le \alpha \). Consequently, it implies that if the i-th test yields \(\Pr (Z_i = 0 \mid {{\varvec{Y}}}_i, {\hat{\pi }}_0) \le \alpha \), it must be included in the rejection set (because \(\Omega \) is the largest such set). Therefore, to prove the corollary, it is sufficient to show that \(\Pr (Z_i=0 \mid {{\varvec{Y}}}_i, {\hat{\pi }}_0) \le \alpha \), provided that \(\mathrm{BF}_i \ge \frac{m}{\alpha }\).
Applying the EBF procedure, a single Bayes factor with the value exceeding m leads to
This implies
\(\square \)
Rights and permissions
About this article
Cite this article
Wen, X. Robust Bayesian FDR Control Using Bayes Factors, with Applications to Multi-tissue eQTL Discovery. Stat Biosci 9, 28–49 (2017). https://doi.org/10.1007/s12561-016-9153-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12561-016-9153-0