Skip to main content
SpringerLink
Log in

From Claude Shannon’s Information Entropy to Spike-Time Data Compression Theory

  • Chapter
  • First Online:
Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence

Part of the book series: Springer Series on Bio- and Neurosystems ((SSBN,volume 7))

  • 2680 Accesses

Abstract

This chapter of the book proposes a new information theory for temporal data compression through spike-time encoding for the purpose of reducing the amount of raw data from time series but preserving the information in terms of accuracy of pattern recognition and pattern classification. Most of the data in information sciences are temporal or spatio/spectro temporal, such as brain data, audio and video data, environmental and ecological data, financial and social data, etc. as discussed in the other chapters of the book and the proposed data compression method is applicable to all of them.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. N. Sengupta, N. Kasabov, Spike-time encoding as a data compression technique for pattern recognition of temporal data. Inf. Sci. 406–407, 133–145 (2017)

    Article  Google Scholar 

  2. E.N. Brown, R.E. Kass, P.P. Mitra, Multiple neural spike train data analysis: state-of-the-art and future challenges. Nat. Neurosci. 7(5), 456–461 (2004)

    Article  Google Scholar 

  3. Z.F. Mainen, T.J. Sejnowski, Reliability of spike timing in neocortical neurons. Science 268(5216), 2003–2006 (1995)

    Article  Google Scholar 

  4. J.H. Maunsell, J.R. Gibson, Visual response latencies in striate cortex of the macaque monkey. J. Neurophysiol. 68(4), 1332–1344 (1992)

    Article  Google Scholar 

  5. T. Gollisch, M. Meister, Rapid neural coding in the retina with relative spike latencies. Science 319(5866), 1108–1111 (2008)

    Article  Google Scholar 

  6. R.M. Hallock, P.M. Di Lorenzo, Temporal coding in the gustatory system. Neurosci. Biobehav. Rev. 30(8), 1145–1160 (2006)

    Article  Google Scholar 

  7. C.E. Shannon, A mathematical theory of communication. ACM SIGMOBILE Mob. Comput. Commun. Rev. 5(1), 3–55 (2001)

    Article  MathSciNet  Google Scholar 

  8. A.N. Kolmogorov, Three approaches to the quantitative definition of information. Probl. Inf. Transm. 1(1), 1–7 (1965)

    MathSciNet  Google Scholar 

  9. G.J. Chaitin, On the length of programs for computing finite binary sequences. J. ACM (JACM) 13(4), 547–569 (1966)

    Article  MathSciNet  Google Scholar 

  10. P. Grunwald, P. Vitányi, Shannon information and kolmogorov complexity, arXiv preprint cs/0410002

    Google Scholar 

  11. H. de Garis, An artificial brain atr’s cam-brain project aims to build/evolve an artificial brain with a million neural net modules inside a trillion cell cellular automata machine. New Gener. Comput. 12(2), 220–221 (1994)

    Article  Google Scholar 

  12. T. Iakymchuk, A. Rosado-Munoz, M. Bataller-Mompean, J. Guerrero-Martinez, J. Frances-Villora, M. Wegrzyn, M. Adamski, Hardware-accelerated spike train generation for neuromorphic image and video processing, in 2014 IX Southern Conference on Programmable Logic (SPL) (IEEE, 2014), pp. 1–6

    Google Scholar 

  13. N. Kasabov, N.M. Scott, E. Tu, S. Marks, N. Sengupta, E. Capecci, M. Othman, M.G. Doborjeh, N. Murli, R. Hartono et al., Evolving spatio-temporal data machines based on the neucube neuromorphic framework: design methodology and selected applications. Neural Netw. 78(2016), 1–14 (2016)

    Article  Google Scholar 

  14. M. Hough, H. De Garis, M. Korkin, F. Gers, N.E. Nawa, Spiker: analog waveform to digital spiketrain conversion in atrs artificial brain (cam-brain) project, in International Conference on Robotics and Artificial Life (Citeseer, 1999)

    Google Scholar 

  15. B. Schrauwen, J. Van Campenhout, BSA, a fast and accurate spike train encoding scheme, in Proceedings of the International Joint Conference on Neural Networks, vol. 4 (IEEE Piscataway, NJ, 2003), pp. 2825–2830

    Google Scholar 

  16. M. Dorigo, V. Maniezzo, A. Colorni, Ant system: optimization by a colony of cooperating agents. IEEE Trans. Syst. Man Cybern. Part B Cybern. 26(1), 29–41 (1996)

    Article  Google Scholar 

  17. H. De Garis, N. E. Nawa, M. Hough, M. Korkin, Evolving an optimal de/convolution function for the neural net modules of atr’s artificial brain project, in International Joint Conference on Neural Networks, 1999. IJCNN’99, vol. 1 (IEEE, 1999), pp. 438–443

    Google Scholar 

  18. N. Sengupta, N. Scott, N. Kasabov, Framework for knowledge driven optimisation based data encoding for brain data modelling using spiking neural network architecture, in Proceedings of the Fifth International Conference on Fuzzy and Neuro Computing sFANCCO-2010) (Springer, 2010), pp. 109–118

    Google Scholar 

  19. F.G. Ashby, Statistical Analysis of fMRI Data (MIT Press, 2011)

    Google Scholar 

  20. M.D. Nunez, P.L. Nunez, R. Srinivasan, Electroencephalography (EEG): neurophysics, experimental methods, and signal processing, in Handbook of Neuroimaging Data Analysis (Chapman & Hall/CRC, 2016) (Chapter)

    Google Scholar 

  21. B. Babu, M. Jehan, Differential evolution for multi-objective optimization, in The 2003 Congress on Evolutionary Computation, 2003. CEC’03, vol. 4 (IEEE, 2003), pp. 2696–2703

    Google Scholar 

  22. L. Yiqing, Y. Xigang, L. Yongjian, An improved pso algorithm for solving non-convex nlp/minlp problems with equality constraints. Comput. Chem. Eng. 31(3), 162–203 (2007)

    Article  Google Scholar 

  23. K. Deb, An efficient constraint handling method for genetic algorithms. Comput. Methods Appl. Mech. Eng. 186(2), 311–338 (2000)

    Article  Google Scholar 

  24. K. Deep, K.P. Singh, M.L. Kansal, C. Mohan, A real coded genetic algorithm for solving integer and mixed integer optimization problems. Appl. Math. Comput. 212(2), 505–518 (2009)

    MathSciNet  MATH  Google Scholar 

  25. G.M. Boynton, S.A. Engel, G.H. Glover, D.J. Heeger, Linear systems analysis of functional magnetic resonance imaging in human v1. J. Neurosci. 16(13), 4207–4221 (1996)

    Article  Google Scholar 

  26. K.J. Friston, O. Josephs, G. Rees, R. Turner, Nonlinear event-related responses in fMRI. Magn. Reson. Med. 39(1), 41–52 (1998)

    Article  Google Scholar 

  27. G.H. Glover, Deconvolution of impulse response in event-related bold fMRI 1. Neuroimage 9(4), 416–429 (1999)

    Article  Google Scholar 

  28. X. Wang, T. Mitchell, Detecting cognitive states using machine learning. Technical report, CMU CALD Technical Report for Summer Work (2002)

    Google Scholar 

  29. L.-N. Do, H.-J. Yang, A robust feature selection method for classification of cognitive states with fMRI data, in Advances in Computer Science and its Applications (Springer, 2014), pp. 71–76

    Google Scholar 

  30. N. Nuntalid, K. Dhoble, N. Kasabov, Eeg classification with BSA spike encoding algorithm and evolving probabilistic spiking neural network, in International Conference on Neural Information Processing (Springer, 2011), pp. 451–460

    Google Scholar 

  31. T.M. Mitchell, R. Hutchinson, M.A. Just, R.S. Niculescu, F. Pereira, X. Wang, Classifying instantaneous cognitive states from fMRI data, in American Medical Informatics Association Annual Symposium (2003)

    Google Scholar 

  32. M.A. Just, S.D. Newman, T.A. Keller, A. McEleney, P.A. Carpenter, Imagery in sentence comprehension: an fMRI study. Neuroimage 21(1), 112–124 (2004)

    Article  Google Scholar 

  33. J.D. Victor, K.P. Purpura, Metric-space analysis of spike trains: theory, algorithms and application. Netw. Comput. Neural Syst. 8(2), 127–164 (1997)

    Article  Google Scholar 

  34. N.K. Kasabov, Neucube: a spiking neural network architecture for mapping, learning and understanding of spatiotemporal brain data. Neural Netw. 52(2014), 62–76 (2014)

    Article  Google Scholar 

  35. N. Kasabov, Springer Handbook of Bio-/Neuroinformatics (Springer, 2014)

    Google Scholar 

  36. N. Sengupta, PhD Thesis, Auckland University of Technology, 2018

    Google Scholar 

Download references

Acknowledgements

Some of the presented material in this chapter was first published in [1]. I would like to acknowledge the significant contribution of Neelava Sengupta for the development of the method from Sect. 21.2 and the experiments in 21.3.

Author information

Authors and Affiliations

  1. Knowledge Engineering and Discovery Research Institute (KEDRI), Auckland University of Technology, Auckland, New Zealand

    Nikola K. Kasabov

Authors
  1. Nikola K. Kasabov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikola K. Kasabov .

Appendix

Appendix

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer-Verlag GmbH Germany, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kasabov, N.K. (2019). From Claude Shannon’s Information Entropy to Spike-Time Data Compression Theory. In: Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence . Springer Series on Bio- and Neurosystems, vol 7. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-57715-8_21

Download citation

Publish with us

Policies and ethics

Search

Navigation