Skip to main content
SpringerLink
Log in

Geometric Deep Learning: A Temperature Based Analysis of Graph Neural Networks

  • Conference paper
  • First Online:
Geometric Science of Information (GSI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14072))

Included in the following conference series:

Abstract

We examine a Geometric Deep Learning model as a thermodynamic system treating the weights as non-quantum and non-relativistic particles. We employ the notion of temperature previously defined in [5] and study it in the various layers for GCN and GAT models. Potential future applications of our findings are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for Boltzmann machines. Cogn. Sci. 9(1), 147–169 (1985)

    Article  Google Scholar 

  2. Barbaresco, F.: Lie group statistics and Lie group machine learning based on Souriau Lie groups thermodynamics and Koszul-Souriau-Fisher metric: new entropy definition as generalized Casimir invariant function in coadjoint representation. Entropy 22(6), 642 (2020)

    Article  MathSciNet  Google Scholar 

  3. Chaudhari, P., Soatto, S.: Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks. In: 2018 Information Theory and Applications Workshop (ITA), pp. 1–10 (2018)

    Google Scholar 

  4. Chaudhari, P., et al.: Entropy-sgd: biasing gradient descent into wide valleys. In: International Conference on Learning Representations ICLR 2017, 1611.01838 (2017)

    Google Scholar 

  5. Fioresi, R., Faglioni, F., Morri, F., Squadrani, L.: On the thermodynamic interpretation of deep learning systems. In: Geometric Science of Information: 5th International Conference (2021)

    Google Scholar 

  6. Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two-state neurons. PNAS 81(10), 3088–3092 (1984)

    Google Scholar 

  7. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106(4), 620 (1957)

    Google Scholar 

  8. Lapenna, M., Faglioni, F., Fioresi, R.: Thermodynamics modeling of deep learning systems. Front. Phys. 11 (2023). https://doi.org/10.3389/fphy.2023.1145156

  9. Marle, C.-M.: From tools in symplectic and Poisson geometry to J. M. Souriau’s theories of statistical mechanics and thermodynamics. Entropy 18(10), 370 (2016)

    Google Scholar 

  10. Salakhutdinov, R., Hinton, G.: Deep Boltzmann machines. In: Artificial Intelligence and Statistics, pp. 440–455. PMLR (2009)

    Google Scholar 

  11. Anahory Simoes, A., De Leon, M., Lainz Valcazar, M., De Diego, D.M.: Contact geometry for simple the rmodynamical systems with friction. Proc. R. Soci. A 476(2241), 20200244 (2020)

    Google Scholar 

  12. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, PMLR 37, pp. 448–456 (2015)

    Google Scholar 

  13. Monti, F., Boscaini, D., Masci, J., Rodolà, E., et al.: Geometric deep learning on graphs and manifolds using mixture model CNNs. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017). https://doi.org/10.1109/CVPR.2018.00335

  14. Gou, M., Xiong, F., Camps, O., Sznaier, M.: MoNet: moments embedding network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1611.08402 (2018)

    Google Scholar 

  15. Avelar, P.H.C., Tavares, A.R., da Silveira, T.L.T., et al.: Superpixel image classification with graph attention networks. In: 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 203–209 (2020). https://doi.org/10.1109/SIBGRAPI51738.2020.00035

  16. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), 21–24 June (2010)

    Google Scholar 

  17. Srivastava, N., Hinton, G., Krizhevsky, A., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  18. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations ICLR 2017 (2017). https://openreview.net/forum?id=SJU4ayYgl

  19. Risken, H.: The Fokker-Planck Equation. In: Methods of Solution and Applications, Springer, Berlin (1996). https://doi.org/10.1007/978-3-642-61544-3

  20. Velikovi, B.Y., et al.: Graph attention networks. In: International Conference on Learning Representations ICLR 2018 (2018). https://doi.org/10.40550/ARXIV.1710.10903

Download references

Author information

Authors and Affiliations

  1. University of Bologna, Bologna, Italy

    M. Lapenna, F. Zanchetta & R. Fioresi

  2. University of Modena, Modena, Italy

    F. Faglioni

Authors
  1. M. Lapenna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. Faglioni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. F. Zanchetta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Fioresi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Lapenna .

Editor information

Editors and Affiliations

  1. Sony Computer Science Laboratories Inc., Tokyo, Japan

    Frank Nielsen

  2. THALES Land and Air Systems, Meudon, France

    Frédéric Barbaresco

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lapenna, M., Faglioni, F., Zanchetta, F., Fioresi, R. (2023). Geometric Deep Learning: A Temperature Based Analysis of Graph Neural Networks. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2023. Lecture Notes in Computer Science, vol 14072. Springer, Cham. https://doi.org/10.1007/978-3-031-38299-4_65

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-38299-4_65

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-38298-7

  • Online ISBN: 978-3-031-38299-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation