Abstract
We provide here a novel algebraic characterization of two information measures associated with a vector-valued random variable, its differential entropy and the dimension of the underlying space, purely based on their recursive properties (the chain rule and the nullity-rank theorem, respectively). More precisely, we compute the information cohomology of Baudot and Bennequin with coefficients in a module of continuous probabilistic functionals over a category that mixes discrete observables and continuous vector-valued observables, characterizing completely the 1-cocycles; evaluated on continuous laws, these cocycles are linear combinations of the differential entropy and the dimension.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aczél, J., Daróczy, Z.: On Measures of Information and Their Characterizations. Mathematics in Science and Engineering. Academic Press, Cambridge (1975)
Baudot, P., Bennequin, D.: The homological nature of entropy. Entropy 17(5), 3253–3318 (2015)
Chang, J.T., Pollard, D.: Conditioning as disintegration. Statistica Neerlandica 51(3), 287–317 (1997)
Devroye, L., Györfi, L.: Nonparametric Density Estimation: The \(L_1\) View. Wiley Series in Probability and Mathematical Statistics. Wiley, Hoboken (1985)
Devroye, L., Györfi, L.: Distribution and density estimation. In: Györfi, L. (ed.) Principles of Nonparametric Learning. ICMS, vol. 434, pp. 211–270. Springer, Vienna (2002). https://doi.org/10.1007/978-3-7091-2568-7_5
Ghourchian, H., Gohari, A., Amini, A.: Existence and continuity of differential entropy for a class of distributions. IEEE Commun. Lett. 21(7), 1469–1472 (2017)
Kolmogorov, A., Shiryayev, A.: Selected Works of A. N. Kolmogorov. Volume III: Information Theory and the Theory of Algorithms. Mathematics and Its Applications, Kluwer Academic Publishers (1993)
Otáhal, A.: Finiteness and continuity of differential entropy. In: Mandl P., Huskovsa, M. (eds.) Asymptotic Statistics, pp. 415–419. Springer, Heidelberg (1994). https://doi.org/10.1007/978-3-642-57984-4_36
Vigneaux, J.P.: Topology of statistical systems: a cohomological approach to information theory. Ph.D. thesis, Université de Paris (2019). https://hal.archives-ouvertes.fr/tel-02951504v1
Vigneaux, J.P.: Information structures and their cohomology. Theory Appl. Categ. 35(38), 1476–1529 (2020)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Vigneaux, J.P. (2021). Information Cohomology of Classical Vector-Valued Observables. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2021. Lecture Notes in Computer Science(), vol 12829. Springer, Cham. https://doi.org/10.1007/978-3-030-80209-7_58
Download citation
DOI: https://doi.org/10.1007/978-3-030-80209-7_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80208-0
Online ISBN: 978-3-030-80209-7
eBook Packages: Computer ScienceComputer Science (R0)