Hostname: page-component-848d4c4894-wzw2p Total loading time: 0 Render date: 2024-05-01T18:38:32.642Z Has data issue: false hasContentIssue false
  • English
  • Français

On the Ziv–Merhav theorem beyond Markovianity I

Part of: Communication, information Approximation methods and numerical treatment of dynamical systems Topological dynamics

Published online by Cambridge University Press:  07 March 2024

Nicholas Barnfield Open the ORCID record for Nicholas Barnfield [Opens in a new window] ,
Raphaël Grondin Open the ORCID record for Raphaël Grondin [Opens in a new window] ,
Gaia Pozzoli Open the ORCID record for Gaia Pozzoli [Opens in a new window]  and
Renaud Raquépas Open the ORCID record for Renaud Raquépas [Opens in a new window]
Nicholas Barnfield
Affiliation:
Department of Mathematics and Statistics, McGill University, Montréal, QC, Canada e-mail: nicholas.barnfield@mail.mcgill.ca
Raphaël Grondin
Affiliation:
Department of Mathematics and Statistics, McGill University, Montréal, QC, Canada e-mail: raphael.grondin@mail.mcgill.ca
Gaia Pozzoli
Affiliation:
Department of Mathematics, CY Cergy Paris Université, CNRS UMR 8088, Cergy-Pontoise, France e-mail: gaia.pozzoli@cyu.fr
Renaud Raquépas*
Affiliation:
Courant Institute of Mathematical Sciences, New York University, New York, NY, United States
*
e-mail: rr4374@nyu.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

We generalize to a broader class of decoupled measures a result of Ziv and Merhav on universal estimation of the specific cross (or relative) entropy, originally for a pair of multilevel Markov measures. Our generalization focuses on abstract decoupling conditions and covers pairs of suitably regular g-measures and pairs of equilibrium measures arising from the “small space of interactions” in mathematical statistical mechanics.

Keywords

Cross entropyestimatorparsingmatch lengthsdecoupling

MSC classification

Primary: 94A17: Measures of information, entropy 37M25: Computational methods for ergodic theory (approximation of invariant measures, computation of Lyapunov exponents, entropy) 37B10: Symbolic dynamics
Type
Article
Information
Canadian Journal of Mathematics , First View , pp. 1 - 25
Check for updates
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of Canadian Mathematical Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The research of N.B. and R.R. was partially funded by the Fonds de recherche du Québec – Nature et technologies (FRQNT) and by the Natural Sciences and Engineering Research Council of Canada (NSERC). The research of R.G. was partially funded by the Rubin Gruber Science Undergraduate Research Award and Axel W Hundemer. The research of G.P. was supported by the CY Initiative of Excellence through the grant Investissements d’Avenir ANR-16-IDEX-0008, and was done under the auspices of the Gruppo Nazionale di Fisica Matematica (GNFM) section of the Istituto Nazionale di Alta Matematica (INdAM) while G.P. was a postdoctoral researcher at the University of Milano-Bicocca (Milan, Italy). Part of this work was done during a stay of the four authors in Neuville-sur-Oise, funded by CY Initiative (grant Investissements d’avenir ANR-16-IDEX-0008).

References

Barbieri, S., Gómez, R., Marcus, B., Meyerovitch, T., and Taati, S., Gibbsian representations of continuous specifications: the theorems of Kozlov and Sullivan revisited . Commun. Math. Phys. 382(2021), 1111–1164.CrossRefGoogle Scholar
Basile, C., Benedetto, D., Caglioti, E., and Degli Esposti, M., An example of mathematical authorship attribution . J. Math. Phys. 49(2008), no. 12, 125211.CrossRefGoogle Scholar
Benedetto, D., Caglioti, E., and Loreto, V., Language trees and zipping . Phys. Rev. Lett. 88(2002), 048702.CrossRefGoogle ScholarPubMed
Benoist, T., Cuneo, N., Jakšić, V., and Pillet, C.-A., On entropy production of repeated quantum measurements II examples . J. Stat. Phys. 182(2021), no. 3, 1–71.CrossRefGoogle Scholar
Benoist, T., Jakšić, V., Pautrat, Y., and Pillet, C.-A., On entropy production of repeated quantum measurements I general theory. Commun. Math. Phys. 357(2018), no. 1, 77–123.CrossRefGoogle Scholar
Berghout, S., Fernández, R., and Verbitskiy, E., On the relation between Gibbs and g-measures . Ergodic Theor. Dyn. Syst. 39(2019), no. 12, 3224–3249.CrossRefGoogle Scholar
Bowen, R., Some systems with unique equilibrium states . Math. Syst. Theor. 8(1974), no. 3, 193–202.CrossRefGoogle Scholar
Bradley, R. C., Basic properties of strong mixing conditions. A survey and some open questions . Probab. Surv. 2(2005), 107–144.CrossRefGoogle Scholar
Chandgotia, N., Han, G., Marcus, B., Meyerovitch, T., and Pavlov, R., One-dimensional Markov random fields, Markov chains and topological Markov fields . Proc. Amer. Math. Soc. 142(2014), no. 1, 227–242.CrossRefGoogle Scholar
Chazottes, J.-R. and Ugalde, E., Projection of Markov measures may be Gibbsian . J. Stat. Phys. 111(2003), no. 5/6, 1245–1272.CrossRefGoogle Scholar
Coutinho, D. P. and Figueiredo, M. A., Information theoretic text classification using the Ziv–Merhav method . In: Marques, J. S., Pérez de la Blanca, N., and Pina, P. (eds.), Pattern recognition and image analysis, Lecture Notes in Computer Science, 3523, Springer, Berlin, 2005, pp. 355–362.CrossRefGoogle Scholar
Coutinho, D. P., Fred, A. L., and Figueiredo, M. A., One-lead ECG-based personal identification using Ziv–Merhav cross parsing . In: 20th international conference on pattern recognition, IEEE, Los Alamitos, 2010, pp. 3858–3861.Google Scholar
Cristadoro, G., Degli Esposti, M., Jakšić, V., and Raquépas, R., On a waiting-time result of Kontoyiannis: mixing or decoupling? Stoch Proc. Appl. 166(2023), 104222.CrossRefGoogle Scholar
Cristadoro, G., Degli Esposti, M., Jakšić, V., and Raquépas, R., Recurrence times, waiting times and universal entropy production estimators . Lett. Math. Phys. 113(2023), no. 1, Article no. 19.CrossRefGoogle Scholar
Cuneo, N., Jakšić, V., Pillet, C.-A., and Shirikyan, A., Large deviations and fluctuation theorem for selectively decoupled measures on shift spaces . Rev. Math. Phys. 31(2019), no. 10, 1950036.CrossRefGoogle Scholar
Cuneo, N. and Raquépas, R., Large deviations of return times and related entropy estimators on shift spaces. Commun. Math. Phys. Preprint, 2023, arXiv:2306.05277 [math.PR].Google Scholar
Denker, M., Grillenberger, C., and Sigmund, K., Ergodic theory on compact spaces, Lecture Notes in Mathematics, 527, Springer, Berlin, 1976.CrossRefGoogle Scholar
Kontoyiannis, I., Asymptotic recurrence and waiting times for stationary processes . J. Theor. Probab. 11(1998), no. 3, 795–811.CrossRefGoogle Scholar
Kontoyiannis, I. and Suhov, Y. M., Prefixes and the entropy rate for long-range sources . In: Kelly, F. P. (ed.), Probability, statistics and optimization: a tribute to Peter whittle, Wiley, New York, 1994.Google Scholar
Kwietniak, D., Łącka, M., and Oprocha, P., A panorama of specification-like properties and their consequences . In: Kolyad, S., Möller, M., Moree, P., and Ward, T. (eds.), Dynamics and numbers, Contemporary Mathematics, 669, American Mathematical Society, Providence, RI, 2016, pp. 155–186.CrossRefGoogle Scholar
Lewis, J. T., Pfister, C.-É., and Sullivan, W. G., Entropy, concentration of probability and conditional limit theorems . Markov Proc. Relat. Fields 1(1995), no. 3, 319–386.Google Scholar
Lippi, M., Montemurro, M. A., Degli Esposti, M., and Cristadoro, G., Natural language statistical features of LSTM-generated texts . IEEE Trans. Neural Netw. Learn. Syst. 30(2019), no. 11, 3326–3337.CrossRefGoogle ScholarPubMed
Olivier, E., Sidorov, N., and Thomas, A., On the Gibbs properties of Bernoulli convolutions related to $\beta$ -numeration in multinacci bases . Monatshefte Math. 145(2005), no. 2, 145–174.CrossRefGoogle Scholar
Pfister, C.-É., Thermodynamical aspects of classical lattice systems . In: Sidoravicius, V. (ed.), In and out of equilibrium: probability with a physics flavor, Progress in Probability, 51, Birkhäuser, 2002, pp. 393–472.CrossRefGoogle Scholar
Pfister, C.-É. and Sullivan, W. G., Asymptotic decoupling and weak Gibbs measures for finite alphabet shift spaces . Nonlinearity 33(2020), no. 9, 4799–4817.CrossRefGoogle Scholar
Ro, S., Guo, B., Shih, A., Phan, T. V., Austin, R. H., Levine, D., Chaikin, P. M., and Martiniani, S., Model-free measurement of local entropy production and extractable work in active matter . Phys. Rev. Lett. 129(2022), no. 22, 220601.CrossRefGoogle ScholarPubMed
Roldán, É. and Parrondo, J. M. R., Entropy production and Kullback–Leibler divergence between stationary trajectories of discrete systems . Phys. Rev. E. 85(2012), 031129.CrossRefGoogle ScholarPubMed
Ruelle, D., Thermodynamic formalism, 2nd ed., Cambridge University Press, Cambridge, 2004.CrossRefGoogle Scholar
Shields, P. C., Waiting times: positive and negative results on the Wyner–Ziv problem . J. Theor. Probab. 6(1993), no. 3, 499–519.CrossRefGoogle Scholar
Shields, P. C., The ergodic theory of discrete sample paths, Graduate Studies in Mathematics, 13, American Mathematical Society, Providence, RI, 1996.CrossRefGoogle Scholar
van Enter, A. C., Fernández, R., and Sokal, A. D., Regularity properties and pathologies of position-space renormalization-group transformations: scope and limitations of Gibbsian theory . J. Stat. Phys. 72(1993), 879–1167.CrossRefGoogle Scholar
Verbitskiy, E., Thermodynamics of hidden Markov processes . In: Marcus, B., Petersen, K., and Weissman, T. (eds.), Entropy of hidden Markov processes and connections to dynamical systems, London Mathematical Society Lecture Note Series, Cambridge University Press, Cambridge, 2011, pp. 258–272.CrossRefGoogle Scholar
Walters, P., Convergence of the Ruelle operator for a function satisfying Bowen’s condition . Trans. Amer. Math. Soc. 353(2001), no. 1, 327–347.CrossRefGoogle Scholar
Walters, P., Regularity conditions and Bernoulli properties of equilibrium states and g-measures . J. Lond. Math. Soc. 71(2005), no. 2, 379–396.CrossRefGoogle Scholar
Wyner, A. D. and Ziv, J., Some asymptotic properties of the entropy of a stationary ergodic data source with applications to data compression . IEEE Int. Symp. Inf. Theory. 35(1989), no. 6, 1250–1258.CrossRefGoogle Scholar
Yoo, J., On factor maps that send Markov measures to Gibbs measures . J. Stat. Phys. 141(2010), no. 6, 1055–1070.CrossRefGoogle Scholar
Ziv, J. and Lempel, A., Compression of individual sequences via variable-rate coding . IEEE Trans. Inf. Theory 24(1978), no. 5, 530–536.CrossRefGoogle Scholar
Ziv, J. and Merhav, N., A measure of relative entropy between individual sequences with application to universal classification . IEEE Trans. Inf. Theory 39(1993), no. 4, 1270–1279.CrossRefGoogle Scholar