Abstract
Factorial Gaussian graphical Models (fGGMs) have recently been proposed for inferring dynamic gene regulatory networks from genomic high-throughput data. In the search for true regulatory relationships amongst the vast space of possible networks, these models allow the imposition of certain restrictions on the dynamic nature of these relationships, such as Markov dependencies of low order – some entries of the precision matrix are a priori zeros – or equal dependency strengths across time lags – some entries of the precision matrix are assumed to be equal. The precision matrix is then estimated by l1-penalized maximum likelihood, imposing a further constraint on the absolute value of its entries, which results in sparse networks. Selecting the optimal sparsity level is a major challenge for this type of approaches. In this paper, we evaluate the performance of a number of model selection criteria for fGGMs by means of two simulated regulatory networks from realistic biological processes. The analysis reveals a good performance of fGGMs in comparison with other methods for inferring dynamic networks and of the KLCV criterion in particular for model selection. Finally, we present an application on a high-resolution time-course microarray data from the Neisseria meningitidis bacterium, a causative agent of life-threatening infections such as meningitis. The methodology described in this paper is implemented in the R package sglasso, freely available at CRAN, http://CRAN.R-project.org/package=sglasso.
Acknowledgments:
We thank Prof. Nigel Saunders from Brunel University London for providing the data used in this analysis.
Appendix
Derivation of
In this section, we derive the estimator of equation 13. Definition (11) can be further simplified using the Karush-Kuhn-Tucker conditions, i.e.
where
and consequently, the generalized degrees-of-freedoms can be defined as
Definition (15) shows that gdf(ρ) depends on two distinct expected values, i.e.
where Y̅ is an independent copy of Y. The identity (16) suggests that the second expected value can be estimated by leave-one-out cross-validation method, i.e.
where
References
Abegaz, F. and E. Wit (2013): “Sparse time series chain graphical models for reconstructing genetic networks,” Biostatistics, 14, 586–599.10.1093/biostatistics/kxt005Search in Google Scholar PubMed
Aderhold, A., D. Husmeier and M. Grzegorczyk (2014): “Statistical inference of regulatory networks for circadian regulation,” Stat. Appl. Genet. Mol. Biol., 13, 227–273.10.1515/sagmb-2013-0051Search in Google Scholar PubMed
Akaike, H. (1973): Information theory and an extension of the maximum likelihood principle. In: B. N. Petrov and F. Czaki, eds., Second International Symposium on Information Theory, Akademiai Kiado, Budapest, 267–281.Search in Google Scholar
Augugliaro, L., A. M. Mineo and E. C. Wit (2013): “Differential geometric least angle regression: a differential geometric approach to sparse generalized linear models,” J. Roy. Statist. Soc. Ser. B, 75, 471–498.10.1111/rssb.12000Search in Google Scholar
Banerjee, O., L. El Ghaoui and A. d’Aspremont (2008): “Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data,” J. Mach. Learn. Res., 9, 485–516.Search in Google Scholar
Bickel, P. J. and E. Levina (2008): “Regularized estimation of large covariance matrices,” Ann. Statist., 36, 199–227.10.1214/009053607000000758Search in Google Scholar
Bühlmann, P. and S. Van De Geer (2011): Statistics for high-dimensional data: methods, theory and applications, Berlin, Heidelberg: Springer.10.1007/978-3-642-20192-9Search in Google Scholar
Efron, B. (1986): “How biased is the apparent error rate of a prediction rule?” J. Amer. Statist. Assoc., 81, 461–470.10.1080/01621459.1986.10478291Search in Google Scholar
Efron, B. (2004): “The estimation of prediction error: covariance penalties and cross-validation,” J. Amer. Statist. Assoc., 99, 619–632.10.1198/016214504000000692Search in Google Scholar
Efron, B., T. Hastie, I. Johnstone and R. Tibshirani (2004): “Least angle regression,” Ann. Statist., 32, 407–499.10.1214/009053604000000067Search in Google Scholar
Fagnocchi, L., E. Pigozzi, V. Scarlato and I. Delany (2012): “In the NadR regulon, adhesins and diverse meningococcal functions are regulated in response to signals in human saliva,” J. Bacteriol., 194, 460–474.10.1128/JB.06161-11Search in Google Scholar PubMed PubMed Central
Foygel, R. and M. Drton (2010): “Extended Bayesian information criteria for gaussian graphical models,” in: Advances in Neural Information Processing Systems, pp. 604–612.Search in Google Scholar
Friedman, J., T. Hastie, H. Höfling and R. Tibshirani (2007): “Pathwise coordinate optimization,” Ann. Appl. Stat., 1, 302–332.10.1214/07-AOAS131Search in Google Scholar
Friedman, J., T. Hastie and R. Tibshirani (2008): “Sparse inverse covariance estimation with the graphical lasso,” Biostatistics, 9, 432–441.10.1093/biostatistics/kxm045Search in Google Scholar PubMed PubMed Central
Friedman, J. H., T. Hastie and R. Tibshirani (2010): “Regularization paths for generalized linear models via coordinate descent,” J. Stat. Softw., 33, 1–22.10.18637/jss.v033.i01Search in Google Scholar
Gao, X., D. Q. Pu, Y. Wu and X. Xu (2012): “Tuning parameter selection for penalized likelihood estimation of Gaussian graphical model,” Statistica Sinica, 22, 1123–1146.Search in Google Scholar
Genco, C. A. and L. M. Wetzler (2010): Neisseria: molecular mechanisms of pathogenesis, Norfolk, UK: Caister Academic Press.Search in Google Scholar
Giuliani, M. M., J. Adu-Bobie, M. Comanducci, B. Aricò, S. Savino, L. Santini, B. Brunelli, S. Bambini, A. Biolchi, B. Capecchi, E. Cartocci, L. Ciucchi, F. Di Marcello, F. Ferlicca, B. Galli, E. Luzzi, V. Masignani, D. Serruto, D. Veggi, M. Contorni, M. Morandi, A. Bartalesi, V. Cinotti, D. Mannucci, F. Titta, E. Ovidi, J. A. Welsch, D. Granoff, R. Rappuoli and M. Pizza (2006): “A universal vaccine for serogroup B meningococcus,” Proc. Natl. Acad. Sci. USA, 103, 10834–10839.10.1073/pnas.0603940103Search in Google Scholar PubMed PubMed Central
Grzegorczyk, M. and D. Husmeier (2011): “Non-homogeneous dynamic Bayesian networks for continuous data,” Mach. Learn., 83, 355–419.10.1007/s10994-010-5230-7Search in Google Scholar
Guo, J., E. Levina, G. Michailidis and J. Zhu (2011): “Joint estimation of multiple graphical models,” Biometrika, 98, 1–15.10.1093/biomet/asq060Search in Google Scholar PubMed PubMed Central
Højsgaard, S. and S. Lauritzen (2008): “Graphical Gaussian models with edge and vertex symmetries,” J. R. Stat. Soc. Series B Stat Methodol., 70, 1005–1027.10.1111/j.1467-9868.2008.00666.xSearch in Google Scholar
Hoops, S., S. Sahle, R. Gauges, C. Lee, J. Pahle, N. Simus, M. Singhal, L. Xu, P. Mendes and U. Kummer (2006): “Copasia complex pathway simulator,” Bioinformatics, 22, 3067–3074.10.1093/bioinformatics/btl485Search in Google Scholar PubMed
Huang, C. Y. and J. E. Ferrell (1996): “Ultrasensitivity in the mitogen-activated protein kinase cascade,” Proc. Natl. Acad. Sci. USA, 93, 10078–10083.10.1073/pnas.93.19.10078Search in Google Scholar PubMed PubMed Central
Jordan, P. and N. Saunders (2009): “Host iron binding proteins acting as niche indicators for Neisseria meningitidis,” PLoS One, 4, e5198.10.1371/journal.pone.0005198Search in Google Scholar PubMed PubMed Central
Leloup, J.-C. and A. Goldbeter (1999): “Chaos and birhythmicity in a model for circadian oscillations of the {PER} and {TIM} proteins in drosophila,” J. Theor. Biol., 198, 445–459.10.1006/jtbi.1999.0924Search in Google Scholar PubMed
Liu, H., K. Roeder and L. Wasserman (2010): “Stability approach to regularization selection (stARS) for high dimensional graphical models,” In: J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel and A. Culotta, eds., Advances in Neural Information Processing Systems 23, Red Hook, NY: Curran Associates, Inc., pp. 1432–1440.Search in Google Scholar
Lysen, S. (2009): Permuted inclusion criterion: a variable selection technique, PhD thesis, University of Pennsylvania.Search in Google Scholar
Meinshausen, N. and P. Bühlmann (2006): “High-dimensional graphs and variable selection with the lasso,” Ann. Statist., 34, 1436–1462.10.1214/009053606000000281Search in Google Scholar
Miller, K. S. (1981): “On the inverse of the sum of matrices,” Mathematics Magazine, 54, 67–72.10.1080/0025570X.1981.11976898Search in Google Scholar
Pizza, M. and R. Rappuoli (2015): “Neisseria meningitidis: pathogenesis and immunity,” Curr. Opin. Microbiol., 23, 68–72.10.1016/j.mib.2014.11.006Search in Google Scholar PubMed
Rhein, R. O. and K. Strimmer (2007): “From correlation to causation networks: a simple approximate learning algorithm and its application to high-dimensional plant gene expression data,” BMC Syst. Biol., 1, 37.10.1186/1752-0509-1-37Search in Google Scholar PubMed PubMed Central
Rothman, A., P. J. Bickel, E. Levina and J. Zhu (2008): “Sparse permutation invariant covariance estimation,” Electron. J. Stat., 2, 494–515.10.1214/08-EJS176Search in Google Scholar
Ryan, K. J., and L. G. Ray (2010): “Influenza, parainfluenza, respiratory syncytial virus, adenovirus and other respiratory viruses, Chap 9,” In: Sherris Medical Microbiology, 5th edn. New York: McGraw Hill, pp. 167–187.Search in Google Scholar
Saunders, N. and J. Davies (2012): “The use of the pan-Neisseria microarray and experimental design for transcriptomics studies of neisseria,” Methods Mol Biol., 799, 295–317.10.1007/978-1-61779-346-2_18Search in Google Scholar PubMed
Schielke, S., C. Huebner, C. Spatz, V. Nägele, N. Ackermann, M. Frosch, O. Kurzai and A. Schubert-Unkmeir (2009): “Expression of the meningococcal adhesin NadA is controlled by a transcriptional regulator of the MarR family,” Mol. Microbiol., 72, 1054–1067.10.1111/j.1365-2958.2009.06710.xSearch in Google Scholar PubMed
Schoen, C., L. Kischkies, J. Elias and B. J. Ampattuu (2014): “Metabolism and virulence in Neisseria meningitidis,” Front. Cell. Infect. Microbiol., 4, 114.10.3389/fcimb.2014.00114Search in Google Scholar PubMed PubMed Central
Schwarz, G. (1978): “Estimating the dimension of a model,” Ann. Statist., 6, 461–464.10.1214/aos/1176344136Search in Google Scholar
Signorelli, M., V. Vinciotti and E. C. Wit (2015): pnea: Parametric Network Enrichment Analysis, URL http://CRAN.R-project.org/package=pnea, r package version 1.2.0.10.1186/s12859-016-1203-6Search in Google Scholar PubMed PubMed Central
Tettelin, H., N. J. Saunders, J. Heidelberg, A. C. Jeffries, K. E. Nelson, J. A. Eisen, K. A. Ketchum, D. W. Hood, J. F. Peden, R. J. Dodson, W. C. Nelson, M. L. Gwinn, R. DeBoy, J. D. Peterson, E. K. Hickey, D. H. Haft, S. L. Salzberg, O. White, R. D. Fleischmann, B. A. Dougherty, T. Mason, A. Ciecko, D. S. Parksey, E. Blair, H. Cittone, E. B. Clark, M. D. Cotton, T. R. Utterback, H. Khouri, H. Qin, J. Vamathevan, J. Gill, V. Scarlato, V. Masignani, M. Pizza, G. Grandi, L. Sun, H. O. Smith, C. M. Fraser, E. R. Moxon, R. Rappuoli and J. Craig Venter (2000): “Complete genome sequence of neisseria meningitidis serogroup B strain MC58,” Science, 287, 1809–1815.10.1126/science.287.5459.1809Search in Google Scholar PubMed
Vujačić, I., A. Abbruzzo and E. Wit (2015): “A computationally fast alternative to cross-validation in penalized Gaussian graphical models,” J. Stat. Comput. Simul., 85, 3628–3640.10.1080/00949655.2014.992020Search in Google Scholar
Wang, C., D. Sun and K. Toh (2010): “Solving log-determinant optimization problems by a Newton-CG primal proximal point algorithm,” SIAM J. Optim., 20, 2994.10.1137/090772514Search in Google Scholar
Wit, E. and A. Abbruzzo (2015): “Factorial graphical models for dynamic networks,” Net. Sci., 3, 37–57.10.1017/nws.2015.2Search in Google Scholar
Wit, E., E. v. d. Heuvel and J.-W. Romeijn (2012): “All models are wrong...: an introduction to model uncertainty,” Statistica Neerlandica, 66, 217–236.10.1111/j.1467-9574.2012.00530.xSearch in Google Scholar
Wu, T. T. and K. Lange (2008): “Coordinate descent algorithms for lasso penalized regression,” Ann. Appl. Statist., 2, 224–244.10.1214/07-AOAS147Search in Google Scholar
Zou, H., T. Hastie and R. Tibshirani (2007): “On the “degrees of freedom” of the lasso,” Ann. Statist., 35, 2173–2192.10.1214/009053607000000127Search in Google Scholar
Supplemental Material:
The online version of this article (DOI: 10.1515/sagmb-2014-0075) offers supplementary material, available to authorized users.
©2016 by De Gruyter
