Skip to main content
SpringerLink
Log in

Quasi-arithmetic Centers, Quasi-arithmetic Mixtures, and the Jensen-Shannon \(\nabla \)-Divergences

  • Conference paper
  • First Online:
Geometric Science of Information (GSI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14071))

Included in the following conference series:

Abstract

We first explain how the information geometry of Bregman manifolds brings a natural generalization of scalar quasi-arithmetic means that we term quasi-arithmetic centers. We study the invariance and equivariance properties of quasi-arithmetic centers from the viewpoint of the Fenchel-Young canonical divergences. Second, we consider statistical quasi-arithmetic mixtures and define generalizations of the Jensen-Shannon divergence according to geodesics induced by affine connections.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The inverse function theorem [10, 11] in multivariable calculus states only the local existence of an inverse continuously differentiable function \(G^{-1}\) for a multivariate function G provided that the Jacobian matrix of G is not singular.

  2. 2.

    The squared Euclidean/Mahalanobis divergence are not metric distances since they fail the triangle inequality.

References

  1. Alić, M., Mond, B., Pečarić, J., Volenec, V.: The arithmetic-geometric-harmonic-mean and related matrix inequalities. Linear Algebra Appl. 264, 55–62 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  2. Amari, S.i.: Differential-geometrical methods in statistics. Lecture Notes on Statistics 28, 1 (1985). https://doi.org/10.1007/978-1-4612-5056-2

  3. Amari, S.i.: Integration of stochastic models by minimizing \(\alpha \)-divergence. Neural Comput. 19(10), 2780–2796 (2007)

    Google Scholar 

  4. Amari, S.: Information Geometry and Its Applications. AMS, vol. 194. Springer, Tokyo (2016). https://doi.org/10.1007/978-4-431-55978-8

    Book  MATH  Google Scholar 

  5. Banerjee, A., Merugu, S., Dhillon, I.S., Ghosh, J., Lafferty, J.: Clustering with Bregman divergences. J. Mach. Learn. Res. 6(10) (2005)

    Google Scholar 

  6. Ben-Tal, A., Charnes, A., Teboulle, M.: Entropic means. J. Math. Anal. Appl. 139(2), 537–551 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bhatia, R., Gaubert, S., Jain, T.: Matrix versions of the Hellinger distance. Lett. Math. Phys. 109(8), 1777–1804 (2019). https://doi.org/10.1007/s11005-019-01156-0

    Article  MathSciNet  MATH  Google Scholar 

  8. Boyd, S., Boyd, S.P., Vandenberghe, L.: Convex optimization. Cambridge University Press (2004)

    Google Scholar 

  9. Bregman, L.M.: The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 7(3), 200–217 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  10. Clarke, F.: On the inverse function theorem. Pac. J. Math. 64(1), 97–102 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  11. Dontchev, A.L., Rockafellar, R.T.: Implicit Functions and Solution Mappings. SMM, Springer, New York (2009). https://doi.org/10.1007/978-0-387-87821-8

    Book  MATH  Google Scholar 

  12. Hardy, G.H., Littlewood, J.E., Pólya, G., Pólya, G.: Inequalities. Cambridge University Press (1952)

    Google Scholar 

  13. Masrani, V., et al.: \(q\)-paths: Generalizing the geometric annealing path using power means. In: Uncertainty in Artificial Intelligence, pp. 1938–1947. PMLR (2021)

    Google Scholar 

  14. Nakajima, N., Ohmoto, T.: The dually flat structure for singular models. Inform. Geometry 4(1), 31–64 (2021). https://doi.org/10.1007/s41884-021-00044-8

    Article  MathSciNet  MATH  Google Scholar 

  15. Nakamura, Y.: Algorithms associated with arithmetic, geometric and harmonic means and integrable systems. J. Comput. Appl. Math. 131(1–2), 161–174 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  16. Niculescu, C.P., Persson, L.-E.: Convex Functions and Their Applications. CBM, Springer, New York (2006). https://doi.org/10.1007/0-387-31077-0

    Book  MATH  Google Scholar 

  17. Nielsen, F.: On the Jensen-Shannon symmetrization of distances relying on abstract means. Entropy 21(5), 485 (2019)

    Article  MathSciNet  Google Scholar 

  18. Nielsen, F.: Statistical divergences between densities of truncated exponential families with nested supports: duo bregman and duo jensen divergences. Entropy 24(3), 421 (2022)

    Article  MathSciNet  Google Scholar 

  19. Nielsen, F.: Beyond scalar quasi-arithmetic means: Quasi-arithmetic averages and quasi-arithmetic mixtures in information geometry (2023)

    Google Scholar 

  20. Nielsen, F., Nock, R., Amari, S.i.: On clustering histograms with \(k\)-means by using mixed \(\alpha \)-divergences. Entropy 16(6), 3273–3301 (2014)

    Google Scholar 

  21. Nock, R., Nielsen, F.: Fitting the smallest enclosing bregman ball. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 649–656. Springer, Heidelberg (2005). https://doi.org/10.1007/11564096_65

    Chapter  Google Scholar 

  22. Ohara, A.: Geodesics for dual connections and means on symmetric cones. Integr. Eqn. Oper. Theory 50, 537–548 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  23. Pálfia, M.: Classification of affine matrix means. arXiv preprint arXiv:1208.5603 (2012)

  24. Rockafellar, R.T.: Conjugates and Legendre transforms of convex functions. Can. J. Math. 19, 200–205 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  25. Rockafellar, R.T.: Convex analysis, vol. 11. Princeton University Press (1997)

    Google Scholar 

  26. Shima, H., Yagi, K.: Geometry of Hessian manifolds. Differential Geom. Appl. 7(3), 277–290 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  27. Zhang, J.: Nonparametric information geometry: from divergence function to referential-representational biduality on statistical manifolds. Entropy 15(12), 5384–5418 (2013)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sony Computer Science Laboratories Inc., Tokyo, Japan

    Frank Nielsen

Authors
  1. Frank Nielsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frank Nielsen .

Editor information

Editors and Affiliations

  1. Sony Computer Science Laboratories Inc., Tokyo, Japan

    Frank Nielsen

  2. THALES Land and Air Systems, Meudon, France

    Frédéric Barbaresco

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nielsen, F. (2023). Quasi-arithmetic Centers, Quasi-arithmetic Mixtures, and the Jensen-Shannon \(\nabla \)-Divergences. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2023. Lecture Notes in Computer Science, vol 14071. Springer, Cham. https://doi.org/10.1007/978-3-031-38271-0_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-38271-0_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-38270-3

  • Online ISBN: 978-3-031-38271-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation