Skip to main content
SpringerLink
Log in

Neural Graph Revealers

  • Conference paper
  • First Online:
Machine Learning for Multimodal Healthcare Data (ML4MHD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14315))

Included in the following conference series:

Abstract

Sparse graph recovery methods work well where the data follows their assumptions, however, they are not always designed for doing downstream probabilistic queries. This limits their adoption to only identifying connections among domain variables. On the other hand, Probabilistic Graphical Models (PGMs) learn an underlying base graph together with a distribution over the variables (nodes). PGM design choices are carefully made such that the inference and sampling algorithms are efficient. This results in certain restrictions and simplifying assumptions. In this work, we propose Neural Graph Revealers (NGRs) which attempt to efficiently merge the sparse graph recovery methods with PGMs into a single flow. The task is to recover a sparse graph showing connections between the features and learn a probability distribution over them at the same time. NGRs use a neural network as a multitask learning framework. We introduce graph-constrained path norm that NGRs leverage to learn a graphical model that captures complex non-linear functional dependencies between features in the form of an undirected sparse graph. NGRs can handle multimodal inputs like images, text, categorical data, embeddings etc. which are not straightforward to incorporate in the existing methods. We show experimental results on data from Gaussian graphical models and a multimodal infant mortality dataset by CDC (Software: https://github.com/harshs27/neural-graph-revealers).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aluru, M., Shrivastava, H., Chockalingam, S.P., Shivakumar, S., Aluru, S.: EnGRaiN: a supervised ensemble learning method for recovery of large-scale gene regulatory networks. Bioinformatics 38, 1312–1319 (2021)

    Article  Google Scholar 

  2. Banerjee, O., Ghaoui, L.E., d’Aspremont, A.: Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Rese. 9, 485–516 (2008)

    MathSciNet  MATH  Google Scholar 

  3. Belilovsky, E., Kastner, K., Varoquaux, G., Blaschko, M.B.: Learning to discover sparse graphical models. In: International Conference on Machine Learning, pp. 440–448. PMLR (2017)

    Google Scholar 

  4. Bhattacharya, S., Rajan, V., Shrivastava, H.: ICU mortality prediction: a classification algorithm for imbalanced datasets. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)

    Google Scholar 

  5. Bhattacharya, S., Rajan, V., Shrivastava, H.: Methods and systems for predicting mortality of a patient, US Patent 10,463,312, 5 November 2019

    Google Scholar 

  6. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., Elhadad, N.: Intelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1721–1730. ACM (2015)

    Google Scholar 

  7. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  MATH  Google Scholar 

  8. Chickering, D.M.: Learning Bayesian networks is NP-complete. In: Fisher, D., Lenz, HJ. (eds.) Learning from Data. Lecture Notes in Statistics, vol. 112, pp. 121–130. Springer, New York (1996). https://doi.org/10.1007/978-1-4612-2404-4_12

  9. Fernández, A., Garcia, S., Herrera, F., Chawla, N.V.: SMOTE for learning from imbalanced data: progress and challenges, marking the 15-year anniversary. J. Artif. Intell. Res. 61, 863–905 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  10. Frankle, J., Carbin, M.: The lottery ticket hypothesis: finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018)

  11. Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)

    Article  MATH  Google Scholar 

  12. Gallicchio, C., Scardapane, S.: Deep randomized neural networks. In: Oneto, L., Navarin, N., Sperduti, A., Anguita, D. (eds.) Recent Trends in Learning From Data. SCI, vol. 896, pp. 43–68. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43883-8_3

    Chapter  Google Scholar 

  13. Gogate, V., Webb, W., Domingos, P.: Learning efficient Markov networks. In: Advances in Neural Information Processing Systems, vol. 23 (2010)

    Google Scholar 

  14. Greenewald, K., Zhou, S., Hero, A., III.: Tensor graphical lasso (TeraLasso). J. R. Stat. Soc. Ser. B (Stat. Methodol.) 81(5), 901–931 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hallac, D., Park, Y., Boyd, S., Leskovec, J.: Network inference via the time-varying graphical lasso. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 205–213 (2017)

    Google Scholar 

  16. Haury, A.C., Mordelet, F., Vera-Licona, P., Vert, J.P.: TIGRESS: trustful inference of gene regulation using stability selection. BMC Syst. Biol. 6(1), 145 (2012)

    Article  Google Scholar 

  17. Heckerman, D., Chickering, D.M., Meek, C., Rounthwaite, R., Kadie, C.: Dependency networks for inference, collaborative filtering, and data visualization. J. Mach. Learn. Res. 1, 49–75 (2001). https://doi.org/10.1162/153244301753344614

    Article  MATH  Google Scholar 

  18. Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995)

    Article  MATH  Google Scholar 

  19. Hsieh, C.J., Sustik, M.A., Dhillon, I.S., Ravikumar, P., et al.: QUIC: quadratic approximation for sparse inverse covariance estimation. J. Mach. Learn. Res. 15(1), 2911–2947 (2014)

    MathSciNet  MATH  Google Scholar 

  20. Imani, S., Shrivastava, H.: tGLAD: a sparse graph recovery based approach for multivariate time series segmentation. In: 8th Workshop on Advanced Analytics and Learning on Temporal Data (AALTD) at ECML-PKDD (2023). https://doi.org/10.48550/arXiv.2303.11647

  21. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press (2009)

    Google Scholar 

  22. Lee, S.I., Ganapathi, V., Koller, D.: Efficient structure learning of Markov networks using \(l_1\)-regularization. In: Advances in Neural Information Processing Systems, vol. 19 (2006)

    Google Scholar 

  23. Lou, Y., Caruana, R., Gehrke, J., Hooker, G.: Accurate intelligible models with pairwise interactions. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 623–631. ACM (2013)

    Google Scholar 

  24. Margolin, A.A., et al.: ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinf. 7, 1–15 (2006)

    Article  Google Scholar 

  25. Moerman, T., et al.: GRNBoost2 and Arboreto: efficient and scalable inference of gene regulatory networks. Bioinformatics 35(12), 2159–2161 (2019)

    Article  Google Scholar 

  26. Pearl, J., Mackenzie, D.: The Book of Why: The New Science of Cause and Effect. Basic Books (2018)

    Google Scholar 

  27. Pu, X., Cao, T., Zhang, X., Dong, X., Chen, S.: Learning to learn graph topologies. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  28. Rajbhandari, S., Shrivastava, H., He, Y.: AntMan: sparse low-rank compression to accelerate RNN inference. arXiv preprint arXiv:1910.01740 (2019)

  29. Ramanujan, V., Wortsman, M., Kembhavi, A., Farhadi, A., Rastegari, M.: What’s hidden in a randomly weighted neural network? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11893–11902 (2020)

    Google Scholar 

  30. Rolfs, B., Rajaratnam, B., Guillot, D., Wong, I., Maleki, A.: Iterative thresholding algorithm for sparse inverse covariance estimation. In: Advances in Neural Information Processing Systems, vol. 25, pp. 1574–1582 (2012)

    Google Scholar 

  31. Van de Sande, B., et al.: A scalable scenic workflow for single-cell gene regulatory network analysis. Nat. Protoc. 15(7), 2247–2276 (2020)

    Article  Google Scholar 

  32. Scutari, M.: Learning Bayesian networks with the bnlearn R package. J. Stat. Softw. 35(3), 1–22 (2010)

    Article  MathSciNet  Google Scholar 

  33. Shrivastava, H.: On using inductive biases for designing deep learning architectures. Ph.D. thesis, Georgia Institute of Technology (2020)

    Google Scholar 

  34. Shrivastava, H., Bart, E., Price, B., Dai, H., Dai, B., Aluru, S.: Cooperative neural networks (CoNN): exploiting prior independence structure for improved classification. arXiv preprint arXiv:1906.00291 (2019)

  35. Shrivastava, H., Chajewska, U.: Methods for recovering conditional independence graphs: a survey. arXiv preprint arXiv:2211.06829 (2022)

  36. Shrivastava, H., Chajewska, U.: Neural graphical models. In: Proceedings of the 17th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU), August 2023. https://doi.org/10.48550/arXiv.2210.00453

  37. Shrivastava, H., Chajewska, U., Abraham, R., Chen, X.: A deep learning approach to recover conditional independence graphs. In: NeurIPS 2022 Workshop: New Frontiers in Graph Learning (2022). https://openreview.net/forum?id=kEwzoI3Am4c

  38. Shrivastava, H., Chajewska, U., Abraham, R., Chen, X.: uGLAD: sparse graph recovery by optimizing deep unrolled networks. arXiv preprint arXiv:2205.11610 (2022)

  39. Shrivastava, H., et al.: GLAD: learning sparse graph recovery. arXiv preprint arXiv:1906.00271 (2019)

  40. Shrivastava, H., Garg, A., Cao, Y., Zhang, Y., Sainath, T.: Echo state speech recognition. In: 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2021, pp. 5669–5673. IEEE (2021)

    Google Scholar 

  41. Shrivastava, H., Huddar, V., Bhattacharya, S., Rajan, V.: Classification with imbalance: a similarity-based method for predicting respiratory failure. In: 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 707–714. IEEE (2015)

    Google Scholar 

  42. Shrivastava, H., Huddar, V., Bhattacharya, S., Rajan, V.: System and method for predicting health condition of a patient. US Patent 11,087,879, 10 August 2021

    Google Scholar 

  43. Shrivastava, H., Zhang, X., Aluru, S., Song, L.: GRNUlar: gene regulatory network reconstruction using unrolled algorithm from single cell RNA-sequencing data. bioRxiv (2020)

    Google Scholar 

  44. Shrivastava, H., Zhang, X., Song, L., Aluru, S.: GRNUlar: a deep learning framework for recovering single-cell gene regulatory networks. J. Comput. Biol. 29(1), 27–44 (2022)

    Article  Google Scholar 

  45. Singh, M., Valtorta, M.: An algorithm for the construction of Bayesian network structures from data. In: Uncertainty in Artificial Intelligence, pp. 259–265. Elsevier (1993)

    Google Scholar 

  46. Städler, N., Bühlmann, P.: Missing values: sparse inverse covariance estimation and an extension to sparse regression. Stat. Comput. 22(1), 219–235 (2012)

    Google Scholar 

  47. United States Department of Health and Human Services (US DHHS), Centers of Disease Control and Prevention (CDC), National Center for Health Statistics (NCHS), Division of Vital Statistics (DVS): Birth Cohort Linked Birth - Infant Death Data Files, 2004–2015, compiled from data provided by the 57 vital statistics jurisdictions through the Vital Statistics Cooperative Program, on CDC WONDER On-line Database. https://www.cdc.gov/nchs/data_access/vitalstatsonline.htm

  48. Vân Anh Huynh-Thu, A.I., Wehenkel, L., Geurts, P.: Inferring regulatory networks from expression data using tree-based methods. PLoS ONE 5(9), e12776 (2010)

    Google Scholar 

  49. Wang, Y., Jang, B., Hero, A.: The Sylvester Graphical Lasso (SyGlasso). In: International Conference on Artificial Intelligence and Statistics, pp. 1943–1953. PMLR (2020)

    Google Scholar 

  50. Williams, D.R.: Beyond Lasso: a survey of nonconvex regularization in Gaussian graphical models (2020)

    Google Scholar 

  51. Yu, J., Smith, V.A., Wang, P.P., Hartemink, A.J., Jarvis, E.D.: Using Bayesian network inference algorithms to recover molecular genetic regulatory networks. In: International Conference on Systems Biology, vol. 2002 (2002)

    Google Scholar 

  52. Yu, Y., Chen, J., Gao, T., Yu, M.: DAG-GNN: DAG structure learning with graph neural networks. In: International Conference on Machine Learning, pp. 7154–7163. PMLR (2019)

    Google Scholar 

  53. Zhang, M., Jiang, S., Cui, Z., Garnett, R., Chen, Y.: D-VAE: a variational autoencoder for directed acyclic graphs. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  54. Zheng, X., Aragam, B., Ravikumar, P.K., Xing, E.P.: DAGs with NO TEARS: Continuous optimization for structure learning. In: Advances in Neural Information Processing Systems, vol. 31, pp. 9472–9483 (2018)

    Google Scholar 

  55. Zheng, X., Dan, C., Aragam, B., Ravikumar, P., Xing, E.: Learning sparse nonparametric DAGs. In: International Conference on Artificial Intelligence and Statistics, pp. 3414–3425. PMLR (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Microsoft Research, Redmond, USA

    Harsh Shrivastava & Urszula Chajewska

Authors
  1. Harsh Shrivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Urszula Chajewska
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Harsh Shrivastava .

Editor information

Editors and Affiliations

  1. Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Bayern, Germany

    Andreas K. Maier

  2. Technical University of Munich, Garching bei München, Bayern, Germany

    Julia A. Schnabel

  3. College of Engineering, University of Wisconsin–Madison, Madison, WI, USA

    Pallavi Tiwari

  4. German Cancer Research Center, Heidelberg, Germany

    Oliver Stegle

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shrivastava, H., Chajewska, U. (2024). Neural Graph Revealers. In: Maier, A.K., Schnabel, J.A., Tiwari, P., Stegle, O. (eds) Machine Learning for Multimodal Healthcare Data. ML4MHD 2023. Lecture Notes in Computer Science, vol 14315. Springer, Cham. https://doi.org/10.1007/978-3-031-47679-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47679-2_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47678-5

  • Online ISBN: 978-3-031-47679-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation