Abstract
The Information Theory (IT) of Fisher and Shannon provides convenient tools for the systematic and unbiased extraction of the chemical interpretation of the known (experimental or calculated) electron distribution in a molecule. A short overview of the basic concepts, relations, and techniques of IT is presented. The Shannon (S) entropy, reflecting the amount of the uncertainty (spread, disorder) contained in the given probability distribution, and the complementary Fisher (F) (intrinsic-accuracy) measure, focusing on the distribution narrowness (order), are introduced. The relative (“cross”) entropy (entropy deficiency, missing information, directed-divergence) concept of Kullback and Leibler (KL), probing the information distance between the compared probability distributions, is presented. Rudiments of the IT descriptors of the communication channels are outlined and applied to the illustrative symmetric binary channel (SBC). The average conditional-entropy (communication noise) and mutual-information (information flow) quantities of information networks are then discussed in a more detail in view of their importance for interpreting the covalent and ionic bond components within the “communication” theory of the chemical bond. The information characteristics or several dependent probability schemes are then briefly summarized and the variational principle for the constrained extremum of the adopted measure of information, called the extreme physical information (EPI) principle, is advocated as a powerful tool for an unbiased assimilation in the optimum probability distribution of the information contained in the relevant constraints and/or references.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abramson N (1963) Information theory and coding. McGraw-Hill, New York
Ash RB (1965) Information theory. Interscience, New York
Aslangul C, Constanciel R, Daudel R, Kottis P (1972) Adv Quant Chem 6:94
Ayers PW (2000b) J Chem Phys 113:10886
Becke AD, Edgecombe KE (1990) J Chem Phys 92:5397
Bernstein RB (1982) Chemical dynamics via molecular beam and laser techniques. Clarendon, Oxford
Brillouin L (1956) Science and information theory. Academic, New York
Callen HB (1962) Thermodynamics: an introduction to the physical theories of equilibrium thermostatics and irreversible thermodynamics. Wiley, New York
Daudel R (1969) The fundamentals of theoretical chemistry. Pergamon, Oxford
Daudel R (1974) The quantum theory of the chemical bond. D. Reidel, Dordrecht
Dirac PAM (1967) The principles of quantum mechanics. Clarendon, Oxford
Esquivel RO, Rodriquez AL, Sagar RP, Hõ M, Smith VH Jr (1996) Phys Rev A 54:259
Fisher RA (1922) Philos Trans R Soc A (Lond) 222:309
Fisher RA (1925) Proc Camb Philos Soc 22:700
Fisher RA (1959) Statistical methods and scientific inference, 2nd edn. Oliver and Boyd, London
Frieden BR (2000) Physics from the Fisher information – a unification. Cambridge University Press, Cambridge
Frieden BR, Soffer BH (2010) Weighted fisher informations, their derivation and use. Phys Lett A (in press)
Gadre SR (1984) Phys Rev A 30:620
Gadre SR (2002) In: Sen KD (ed) Reviews of modern quantum chemistry: a celebration of the contributions of Robert G. Parr, vol 1. World Scientific, Singapore, p 108
Gadre SR, Bendale RD (1985) Int J Quantum Chem 28:311
Gadre SR, Sears SB (1979) J Chem Phys 71:4321
Gadre SR, Bendale RD, Gejii SP (1985a) Chem Phys Lett 117:138
Gadre SR, Sears SB, Chakravorty SJ, Bendale RD (1985b) Phys Rev A 32:2602
Gopinathan MS, Jug K (1983) Theor Chim Acta (Berl.) 63:497, 511
Hartley RVL (1928) Bell Syst Tech J 7:535
Hirshfeld FL (1977) Theor Chim Acta (Berl) 44:129
Jaynes ET (1957a) Phys Rev 106:620
Jaynes ET (1957b) Phys Rev 108:171
Jaynes ET (1985) In: Smith CR, Grandy WT (eds) Maximum entropy and Bayesian methods in inverse problems. Reidel, Dordrecht
Jug K, Gopinathan MS (1990) In: Maksić ZB (ed) Theoretical models of chemical bonding, vol 2. Springer, Heidelberg, p 77
Khinchin AI (1957) Mathematical foundations of the information theory. Dover, New York
Kullback S (1959) Information theory and statistics. Wiley, New York
Kullback S, Leibler RA (1951) Ann Math Stat 22:79
Mathai AM, Rathie PM (1975) Basic concepts in information theory and statistics: axiomatic foundations and applications. Wiley, New York
Mayer I (1983) Chem Phys Lett 97:270
Mayer I (1985) Theor Chim Acta (Berl.) 67:315
Nagy Á, Parr RG (2000) J Mol Struct THEOCHEM 501:101
Nagy Á, Parr RG (1994) Proc Indian Acad Sci Chem Sci 106:217
Nagy Á, Parr RG (1996) Int J Quantum Chem 58:323
Nalewajski RF (2000c) J Phys Chem A 104:11940
Nalewajski RF (2002a) Phys Chem Chem Phys 4:1710
Nalewajski RF (2002b) Int J Mol Sci 3:237
Nalewajski RF (2002c) Acta Chim Phys Debr 34–35:131
Nalewajski RF (2003b) J Phys Chem A 107:3792
Nalewajski RF (2003d) Chem Phys Lett 375:196
Nalewajski RF (2004a) Ann Phys (Leipzig) 13:201
Nalewajski RF (2004b) Chem Phys Lett 386:265
Nalewajski RF (2004c) Mol Phys 102:531
Nalewajski RF (2004d) Mol Phys 102:547
Nalewajski RF (2004e) Struct Chem 15:395
Nalewajski RF (2005a) Theor Chem Acc 114:4
Nalewajski RF (2005b) Mol Phys 103:451
Nalewajski RF (2005c) J Math Chem 38:43
Nalewajski RF (2006a) Mol Phys 104:365
Nalewajski RF (2006b) Mol Phys 104:493
Nalewajski RF (2006c) Mol Phys 104:1977
Nalewajski RF (2006d) Mol Phys 104:2533
Nalewajski RF (2006e) Mol Phys 104:3339
Nalewajski RF (2006f) Adv Quant Chem 51:235
Nalewajski RF (2006g) Information theory of molecular systems. Elsevier, Amsterdam
Nalewajski RF (2006h) Mol Phys 104:255
Nalewajski RF (2007) J Phys Chem A 111:4855
Nalewajski RF (2008a) J Math Chem 43:265
Nalewajski RF (2008b) J Math Chem 43:780
Nalewajski RF (2008e) Int J Quantum Chem 108:2230
Nalewajski RF (2009a) In: Chattaraj PK (ed) Chemical reactivity theory: a density functional view. Taylor and Francis, London, p 453
Nalewajski RF (2009b) J Math Chem 45:709
Nalewajski RF (2009c) J Math Chem 45:776
Nalewajski RF (2009d) J Math Chem 45:1041
Nalewajski RF (2009e) Int J Quantum Chem 109:425
Nalewajski RF (2009f) Int J Quantum Chem 109:2495
Nalewajski RF (2009g) Adv Quant Chem 56:217
Nalewajski RF (2010a) J Math Chem 47:667
Nalewajski RF (2010b) J Math Chem 47:692
Nalewajski RF (2010c) J Math Chem 47:709
Nalewajski RF (2010d) J Math Chem 47:808
Nalewajski RF (2010f) Information origins of the chemical bond. Nova Science, New York
Nalewajski RF (2010h) Information perspective on molecular electronic structure. In: Mathematical chemistry. Nova Science, New York (in press)
Nalewajski RF (2010i) Information tools for probing chemical bonds. In: Putz M (ed) Chemical information and computation challenges in 21st: a celebration of 2011 international year of chemistry. Nova Science, New York (in press)
Nalewajski RF, Broniatowska E (2003a) J Phys Chem A 107:6270
Nalewajski RF, Broniatowska E (2005) Int J Quantum Chem 101:349
Nalewajski RF, Broniatowska E (2007) Theor Chem Acc 117:7
Nalewajski RF, Jug K (2002) In: Sen KD (ed) Reviews of modern quantum chemistry: a celebration of the contributions of Robert G. Parr, vol 1. World Scientific, Singapore, p 148
Nalewajski RF, Mrozek J (1994) Int J Quantum Chem 51:187
Nalewajski RF, Mrozek J (1996) Int J Quantum Chem 57:377
Nalewajski RF, Parr RG (2000) Proc Natl Acad Sci USA 97:8879
Nalewajski RF, Parr RG (2001) J Phys Chem A 105:7391
Nalewajski RF, Köster AM, Jug K (1993) Theor Chim Acta (Berl.) 85:463
Nalewajski RF, Formosinho SJ, Varandas AJC, Mrozek J (1994a) Int J Quantum Chem 52:1153
Nalewajski RF, Korchowiec J, Michalak A (1994b) Proc Indian Acad Sci Chem Sci 106:353
Nalewajski RF, Korchowiec J, Michalak A (1996a) Top Curr Chem 183:25
Nalewajski RF, Mrozek J, Mazur G (1996b) Can J Chem 100:1121
Nalewajski RF, Mrozek J, Michalak A (1997) Int J Quantum Chem 61:589
Nalewajski RF, Świtka E, Michalak A (2002) Int J Quantum Chem 87:198
Nalewajski RF, Köster AM, Escalante S (2005) J Phys Chem A 109:10038
Nalewajski RF, de Silva P, Mrozek J (2010a) Kinetic-energy/Fisher-information indicators of chemical bonds. In: Wang A, Wesołowski TA (eds) Kinetic energy functional. World Scientific, Singapore (in press)
Nalewajski RF, de Silva P, Mrozek J (2010b) J Mol Struct THEOCHEM 954:57
Nalewajski RF, Szczepanik D, Mrozek J (2011) Adv Quant Chem 68:1
Parr RG, Ayers PW, Nalewajski RF (2005) J Phys Chem A 109:3957
Pfeiffer PE (1978) Concepts of probability theory. Dover, New York
Savin A, Nesper R, Wengert S, Fässler TF (1997) Angew Chem Int Ed Engl 36:1808
Sears SB (1980) PhD Thesis, The University of North Carolina, Chapel Hill
Sears SB, Parr RG, Dinur U (1980) Israel J Chem 19:165
Shannon CE (1948) Bell Syst Technol J 27:379, 623
Shannon CE, Weaver W (1949) The mathematical theory of communication. University of Illinois, Urbana
Silvi B, Savin A (1994) Nature 371:683
Tisza L (1977) Generalized thermodynamics. MIT Press, Cambridge
von Weizsäcker CF (1935) Z Phys 96:431
Wiberg KB (1968) Tetrahedron 24:1083
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Nalewajski, R.F. (2011). Elements of Information Theory. In: Perspectives in Electronic Structure Theory. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20180-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-20180-6_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20179-0
Online ISBN: 978-3-642-20180-6
eBook Packages: Chemistry and Materials ScienceChemistry and Material Science (R0)