Skip to main content
SpringerLink
Log in

A Model-Based Approach for the Analysis of Neuronal Information Transmission in Multi-Input and -Output Systems

  • Published:
Journal of Computational Neuroscience Aims and scope Submit manuscript

Abstract

We present a new method to characterize multi-input and output neuronal systems using information theory. To obtain a lower bound of transinformation we take three steps: (1) Estimation of the deterministic response to isolate components carrying stimulus information. The deviation of the original response from the deterministic estimate is defined as noise. (2) Coordinate transformation using PCA yields an uncorrelated representation. (3) Partial transinformation values are calculated independently either by Shannon's formula assuming normality or based on density estimation for arbitrary distributions. We investigate the performance of the algorithms using simulated data and discuss suitable parameter settings. The approach allows to evaluate the degree to which stimulus features are encoded. Its potential is illustrated by analyses of neuronal activity in cat primary visual cortex evoked by electrical retina stimulation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bercher JF, Vignat C (2000) Estimating the entropy of a signal with applications. IEEE Trans. Signal. Process. 48: 1687–1694.

    Google Scholar 

  • Bialek W, Rieke F, de Ruyter van Steveninck RR, Warland D (1991) Reading a neural code. Science 252: 1854–1857.

    Google Scholar 

  • Bode HW, Shannon CE (1950) A simplified derivation of linear least square smoothing and prediction theory. Proc. I. R. E. 38: 417–425.

    Google Scholar 

  • Borst A, Theunissen FE (1999) Information theory and neural coding. Nat. Neurosci. 2: 947–957.

    Google Scholar 

  • Brenner N, Bialek W, de Ruyter van Steveninck R (2000) Adaptive rescaling maximizes information transmission. Neuron 26: 695–702.

    Google Scholar 

  • Buracas GT, Albright TD (1999) Gauging sensory representation in the brain. Trends Neurosci. 22: 303–309.

    Google Scholar 

  • Buracas GT, Zador AM, DeWeese MR, Albright TD (1998) Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron 20: 959–969.

    Google Scholar 

  • Comon P (1994) Independent component analysis, a new concept? Signal Process. 36: 287–314.

    Google Scholar 

  • Cover TM, Thomas JA(1991) Elements of Information Theory. John Wiley and Sons, Inc., New York.

    Google Scholar 

  • deCharms RC, Zador A(2000) Neural representation and the cortical code. Annu. Rev. Neurosci. 23: 613–647.

    Google Scholar 

  • Deco G, Obradovic D (1997) An Information-Theoretic Approach to Neural Computing. Springer, Berlin.

    Google Scholar 

  • de Ruyter van Steveninck RR, Laughlin SB (1996) The rate of information transfer at graded-potential synapses. Nature 379: 642–645.

    Google Scholar 

  • de Ruyter van Steveninck RR, Lewen GD, Strong SP, Koberle R, Bialek W (1997) Reproducibility and variability in neural spike trains. Science 275: 1805–1808.

    Google Scholar 

  • Dimitrov AG, Miller JP (2001) Neural coding and decoding: Communication channels and quantization. Network 12: 441–472.

    Google Scholar 

  • Dimitrov AG, Miller JP, Aldworth Z, Parker AE (submitted) Spike pattern-based coding schemes in the cricket cercal sensory system. Neurocomputing.

  • Eckhorn R, Grüsser OJ, Kröller J, Pellnitz K, Pöpel B (1976) Effi-ciency of different neuronal codes: Information transfer calculations for three different neuronal systems. Biol. Cybern. 22: 49–60.

    Google Scholar 

  • Eckhorn R, Pöpel B (1974) Rigorous and extended application of information theory to the afferent visual system of the cat. I. Basic concepts. Kybernetik 16: 191–200.

    Google Scholar 

  • Eckhorn R, Pöpel B (1975) Rigorous and extended application of information theory to the afferent visual system of the cat. II. Experimental results. Biol. Cybern. 17: 7–17.

    Google Scholar 

  • Eckhorn R, Pöpel B (1981) Responses of cat retinal ganglion cells to the random motion of a spot stimulus. Vision Res. 21: 435–443.

    Google Scholar 

  • Eckhorn R, Querfurth H(1985) Information transmission by isolated frog muscle spindle. Biol. Cybern. 52: 165–176.

    Google Scholar 

  • Efron B (1994) The Jackknife, the Bootstrap and Other Resampling Plans. 6th edn. Society for Industrial and Applied Mathematics, Philadelphia.

    Google Scholar 

  • Efron B, Tibshirani R (1991) Statistical data analysis in the computer age. Science 253: 390–395.

    Google Scholar 

  • Eger M, Eckhorn R (in revision) Assessing the encoding of stimulus attributes with rapid sequences of stimulus events.

  • Eger M, Eckhorn R, Schanze T (2001) Methods of calculating retinocortical information transmission for multi-dimensional visual and electrical stimuli in the cat's visual system. In Göttingen Neurobiology Report. Proc. 28th Göttingen Neurobiology Conference (Abstract no. 583).

  • Gershon ED, Wiener MC, Latham PE, Richmond BJ (1998) Coding strategies in monkey V1 and inferior temporal cortices. J. Neurophysiol. 79: 1135–1144.

    Google Scholar 

  • Glaser EM, Ruchkin DS (1976) Principles of Neurobiological Signal Analysis. 1st edn. Academic Press, New York.

    Google Scholar 

  • Gray RM (2001) Toeplitz and circulant matrices: A review. http://ee. stanford.edu~gray/toeplitz.pdf.

  • Hinich M(1982) Testing for Gaussianity and linearity of a stationary time series. J. Time Ser. Anal. 3: 169–176.

    Google Scholar 

  • Ihaka R, Gentleman R(1996)Alanguage for data analysis and graphics. J. Comput. Graph. Statist. 5: 299–314.

    Google Scholar 

  • Karhunen J, Wang L, Vigario R (1995) Nonlinear pca type approaches for source separation and independent component analysis. In IEEE Proceedings of ICNN'95.

  • Krausz HI (1974) Identification of nonlinear systems using random impulse train inputs. Biol. Cybern. 19: 217–230.

    Google Scholar 

  • Lee YW, Schetzen M (1965) Measurement of the Wiener kernels of a non-linear system by cross-correlation. Int. J. Contr. 2: 237–255.

    Google Scholar 

  • Linsker R (1988) Self-organization in a perceptual network. IEEE Computer 21: 105–117.

    Google Scholar 

  • Loader CR (1997) Locfit: An Introduction. Statistical Computing and Graphics Newsletter.

  • Loader C (1999) Local Regression and Likelihood. Statistics and Computing. 1st edn. Springer.

  • MacKay DM, McCulloch WS (1952) The limiting information capacity of a neuronal link. Bulletin of Mathematical Biophysics 14: 127–135.

    Google Scholar 

  • Makhoul J (1981) On the eigenvectors of symmetric Toeplitz matrices. IEEE Trans. Acoust. Speech Signal Process. 29: 868–872.

    Google Scholar 

  • Marmarelis PZ, McCann GD (1973) Development and application of white-noise modeling techniques for studies of insect visual nervous system. Biol. Cybern. 12: 74–89.

    Google Scholar 

  • Moddemeijer R (1989) On estimation of entropy and mutual information of continuous distributions. Signal Process. 16: 233–248.

    Google Scholar 

  • Moddemeijer R (1999) A statistic to estimate the variance of the histogram-based mutual information estimator based on dependent pairs of observations. Signal Process. 75: 51–63.

    Google Scholar 

  • Oppenheim AV, Schafer RW(1989) Discrete-time signal processing. Prentice-Hall, Englewood Cliffs, NJ.

    Google Scholar 

  • Optican LM, Richmond BJ (1987) Temporal encoding of twodimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. J. Neurophysiol. 57: 162–178.

    Google Scholar 

  • Palm G, Pöpel B(1985)Volterra representation and Wiener-like identification of nonlinear systems: Scope and limitations. Quarterly Review of Biophysics 18: 135–164.

    Google Scholar 

  • Plumbley MD(1991)Oninformation theory and unsupervised neural networks. Technical report, Cambridge University Engineering Department, UK.

  • Priestley MB (1980) State-dependent models: A general approach to non-linear time series analysis. J. Time Ser. Anal. 1: 47–71.

    Google Scholar 

  • Rao TS, Gabr MM(1980)Atest for linearity of stationary time series. J. Time Ser. Anal. 1: 145–158.

    Google Scholar 

  • Reich DS, Mechler F, Victor JD (2000) Formal and attribute-specific information in primary visual cortex. J. Neurophysiol. 85: 305–318.

    Google Scholar 

  • Reinagel P, Reid RC (2000) Temporal coding of visual information in the thalamus. J. Neurosci. 20: 5392–5400.

    Google Scholar 

  • Richmond BJ, Optican LM, Podell M, Spitzer H (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. I. Response characteristics. J. Neurophysiol. 57: 132–146.

    Google Scholar 

  • Rieke F, Bodnar DA, Bialek W (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proc. R. Soc. Lond. B. Biol. Sci. 262: 259–265.

    Google Scholar 

  • Rieke F, Warland DK, de Ruyter van Steveninck RR, Bialek W(1998) Spikes: Exploring the Neural Code. 1st edn. The MIT Press, Cambridge.

    Google Scholar 

  • Rife DC, Boorstyn RR (1974) Single-tone parameter estimation from discrete-time observations. IEEE Trans. Inf. Theory 20: 591–598.

    Google Scholar 

  • Roddey JC, Girish B, Miller JP (2000) Assessing the performance of neural encoding models in the presence of noise. J. Comput. Neurosci. 8: 95–112.

    Google Scholar 

  • Shannon CE (1948) A mathematical theory of communication. The Bell Systems Technical Journal 27: 379–423, 623- 656.

    Google Scholar 

  • Shannon CE (1949) Communication in the presence of noise. Proc. I. R. E. 37: 10–21.

    Google Scholar 

  • Snedecor GW, Cochran WG (1989) Statistical Methods. 8th edn. Iowa State University Press.

  • Stein RB (1967) The information capacity of nerve cells using a frequency code. Biophys. J. 7: 797–825.

    Google Scholar 

  • Strong SP, Koberle R, de Ruyter van Steveninck R, Bialek W(1998) Entropy and information in neural spike trains. Phys. Rev. Lett. 80: 197–200.

    Google Scholar 

  • Theunissen FE (1993) An Investigation of Sensory Coding Principles Using Advanced Statistical Techniques. Ph.D. Thesis, University of California at Berkeley, Berkeley.

    Google Scholar 

  • Theunissen FE, David SV, Singh NC, Hsu A, Vinje WE, Gallant JL (2001) Estimating spatio-temporal receptive fields of auditory and visual neurons from their responses to natural stimuli. Network 12: 289–316.

    Google Scholar 

  • Unser M (1984) On the approximation of the discrete Karhunenloeve transform for stationary processes. Signal Process. 7: 231–249.

    Google Scholar 

  • Venables WN, Ripley BN (1999) Modern Applied Statistics with S-Plus. Statistics and Computing. 3rd edn. Springer, Berlin.

    Google Scholar 

  • Warland DK, Reinagel P, Meister M(1997) Decoding visual information from a population of retinal ganglion cells. J. Neurophysiol. 78: 2336–2350.

    Google Scholar 

  • Werner G, Mountcastle VB (1965) Neural activity in mechanoreceptive cutaneous afferents: Stimulus-response relations,Weber functions, and information transmission. J. Neurophysiol. 28: 359–397.

    Google Scholar 

  • Wiener MC, Richmond BJ (1999) Using response models to estimate channel capacity for neuronal classification of stationary visual stimuli using temporal coding. J. Neurophysiol. 82: 2861–2875.

    Google Scholar 

  • Wiener N(1950) Extrapolation, Interpolation, and Smoothing of Stationary Time Series. The Technology Press of MIT and JohnWiley & Sons.

  • Wiener N (1958) Nonlinear Problems in Random Theory. The Technology Press of MIT and John Wiley & Sons.

  • Yang HH, Amari S, Cichocki A (1998) Information theoretic approach to blind separation of sources in non-linear mixture. Signal Process. 64: 291–300.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Physics, Neurophysics Group, Philipps-University, Renthof 7, 35032, Marburg, Germany

    M. Eger & R. Eckhorn

Authors
  1. M. Eger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Eckhorn
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Eger, M., Eckhorn, R. A Model-Based Approach for the Analysis of Neuronal Information Transmission in Multi-Input and -Output Systems. J Comput Neurosci 12, 175–200 (2002). https://doi.org/10.1023/A:1016583328930

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1016583328930

Search

Navigation