Learning environmental clues in the KIV model of the cortico-hippocampal formation
Introduction
K sets represent a family of models of increasing complexity that describe various aspects of functioning of vertebrate brains [5]. K models provide a biologically plausible simulation of chaotic spatio-temporal neural processes at the mesoscopic and macroscopic scales. KO is an elementary building block describing second-order dynamics of neural populations. KI is a layer of excitatory or inhibitory KO units, while KII is a double layer of excitatory and inhibitory units. KIII is a set of two or more KII units connected by feedforward and delayed feedback connections [3], [6]. The hippocampal formation (HF) and the sensory cortex are examples of KIII sets. Finally, KIV consists of two or more KIII sets with additional KII and KI [7].
KIII-based modeling of the olfactory system is applied to classify linearly non-separable patterns. The model's performance is compared with those of statistical classification methods and multi-layer feed-forward neural network-based classifications. KIII compares favorably with these methods regarding robustness and noise-tolerance of the pattern recognition, especially for classification of objects that are not linearly separable by any set of features [6].
In this work, the cortex and the hippocampal formation are modeled as KIII sets. This approach creates the basis of their integration at the KIV level, which allows a unified description of spatio-temporal neural dynamics during sensory processing and decision-making. Results obtained by a simplified KIV model for goal-oriented action are given in this work.
Section snippets
KIII models of the cortex and hippocampal formation
In a simplified KIV model, we incorporate two KIII systems. One is for sensory data in the sensory cortex, while the other KIII unit models the hippocampal formation. Fig. 1 illustrates schematically the relationship between the sensory cortex and HF as KIII units. Fig. 1 is a simplified version of the KIV structure [7], and it shows only two KIII sets. In our approach, the external data to HF originate from orientation beacons, while sensory cortices receive visual, infra, etc. signals from
Learning in the hippocampal and cortical KIII models
The hippocampus is strongly involved in navigation [2]. It has two behavioral modes, one is characterized with periodic, and one with aperiodic dynamics. A hippocampus-related navigating algorithm based on reinforcement learning was suggested by Foster et al. [4]. However, their algorithm does not incorporate aperiodic dynamics. Periodic–aperiodic transitions appear in hippocampal place cells. The activity of a place cell is maximum only when the animal is visiting a particular location of its
Multiple T-maze simulations using the cortico-hippocampal KIV set
Here we summarize results obtained by reinforcement learning implemented using Hebbian correlation rule in CA1 and PC. A key component of our approach is the introduction of a periodicity in learning that simulates the theta rhythm. The theta rhythm will be introduced in the numerical experiments by providing the various KIII units with sensory stimuli periodically, at rates corresponding to the theta frequency. We can simulate the theta sampling in computer experiments with the KIV model
Conclusions
In this work, we have introduced a novel method of navigation using the KIV set with hippocampal and cortical parts. We have shown the feasibility of the proposed methodology, and showed that K models are promising dynamic neural networks to address navigation tasks. Our results clearly demonstrate that the applied reinforcement learning algorithm in KIV produces significant learning gains, which are converted into improved navigation of the simulated robot through the environment.
Acknowledgements
This research was supported by grants from NASA, IS NCC-2-1244, and by NSF EIA-0130352.
Robert Kozma holds a Ph.D. in Applied Physics from Delft University of Technology (1992). Currently he is Professor of Computer Science at the University of Memphis. Previously, he has been with the faculty of Tohoku University, Sendai, Japan (1993–1996); Otago University, Dunedin, New Zealand (1996–1998); he held a joint appointment at the Division of Neuroscience and Department of EECS at UC Berkeley (1998–2000). His research focuses at spatio-temporal neurodynamics, multi-sensory fusion
References (8)
- et al.
Parameter optimization in models of the olfactory system
Neural Networks
(1996) - et al.
The KIV model—nonlinear spatio-temporal dynamics of the primordaial vertebrate forebrain
Neurocomputing
(2003) The hierarchy of K sets—from pattern recognition to navigation
(2002)- et al.
Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity
Biol. Cybernet.
(2000)
Cited by (7)
Managing complexity: from visual perception to sustainable transitions—contributions of Brunswik’s Theory of Probabilistic Functionalism
2017, Environment Systems and DecisionsForward and backward modeling: From single cells to neural population and back
2016, Studies in Systems, Decision and ControlModeling hippocampal theta oscillation: Applications in neuropharmacology and robot navigation
2006, International Journal of Intelligent SystemsApplying KIV dynamic neural network model for real time navigation by mobile robot EMMA
2004, IEEE International Conference on Neural Networks - Conference Proceedings
Robert Kozma holds a Ph.D. in Applied Physics from Delft University of Technology (1992). Currently he is Professor of Computer Science at the University of Memphis. Previously, he has been with the faculty of Tohoku University, Sendai, Japan (1993–1996); Otago University, Dunedin, New Zealand (1996–1998); he held a joint appointment at the Division of Neuroscience and Department of EECS at UC Berkeley (1998–2000). His research focuses at spatio-temporal neurodynamics, multi-sensory fusion using chaotic neural nets, and self-organized development of intelligent behavior in animals and animats.
Walter J Freeman studied physics and mathematics at MIT, Philosophy at the University of Chicago, Medicine at Yale University (MD cum laude 1954), internal medicine at Johns Hopkins, and Neurophysiology at UCLA. He has taught Brain Science in the University of California at Berkeley since 1959, where he is Professor of the Graduate School. He received the Pioneer Award from the Neural Networks Council of the IEEE, and he is IEEE Fellow. He is the author of >350 articles and four books: “Mass Action in the Nervous System” (1975), “Societies of Brains” (1995), “How Brains Make Up Their Minds” (1999), “Neurodynamics: An Exploration of Mesoscopic Brain Dynamics” (2000).
Péter Érdi is the Henry R. Luce Professor in Kalamazoo College. He also the head of Department of Biophysics, KFKI Research Institute for Particle and Nuclear Physics, Hungarian Academy of Sciences, and Széchenyi Professor at the Eötvös University. His main scientific interest is the computational approach to the functional organization of the hippocamal formation. He is the co-author with Arbib and Szentagothai of the popular volume Neural Organization—Structure, Function, and Dynamics (1997).
Derek Wong completed his undergraduate studies in Computer Science in 2002, and currently he is working for an MSc degree at the Computer Science Division, The University of Memphis. He is a research assistant in the Computational Neurodynamics Lab directed by Dr. Kozma. He is also with the Cognitive and Linguistic Systems Lab at the Department of Psychology.