Abstract

Nonextensivity in a memory network access mechanism

Roseli S. Wedemann^I,* * Electronic address: roseli@ime.uerj.br ; Raul Donangelo^II,† † Electronic address: donangel@if.ufrj.br ; Luís A. V. de Carvalho^III,‡ ‡ Electronic address: LuisAlfredo@ufrj.br

^IInstituto de Matemática e Estatística, Universidade do Estado do Rio de Janeiro, Rua São Francisco Xavier 524, 20550-900, Rio de Janeiro, Brazil

^IIInstituto de Física, Universidade Federal do Rio de Janeiro, Caixa Postal 68528, 21941-972, Rio de Janeiro, Brazil

^IIIPrograma de Engenharia de Sistemas e Computação, Universidade Federal do Rio de Janeiro, Caixa Postal 68511, 21945-970, Rio de Janeiro, Brazil

ABSTRACT

We have previously described neurotic psychopathology and psychoanalytic working-through by an associative memory mechanism, based on a neural network. Memory was initially modelled by a Boltzmann machine (BM). We simulated known microscopic mechanisms that control synaptic properties and showed that the network self-organizes to a hierarchical, clustered structure. Some properties of the complex networks which result from this self-organization indicate that a generalization of the BM may be necessary to model memory. We have thus generalized the memory model, using Generalized Simulated Annealing, derived from the nonextensive formalism, and show some properties of the resulting memory access mechanism.

Keywords: Consciousness-unconsciousness, Neuroses, Self-organized neural networks, Boltzmann machine, Generalized simulated annealing, Hierarchical memory model

1. INTRODUCTION

Early psychoanalytic research regarding the neuroses found that traumatic and repressed memories are knowledge which is present in the subject, but which is symbolically inaccessible to him. It is therefore considered unconscious knowledge [1, 2]. Freud observed that neurotic patients systematically repeated symptoms in the form of ideas and impulses, and called this tendency a compulsion to repeat [3], which he related to repressed or traumatic memory traces [1].

Neurotic analysands have obtained relief and cure of painful symptoms through a mechanism called working-through. This procedure aims at developing knowledge regarding the causes of symptoms by accessing unconscious memories, and understanding and changing the analysand's compulsion to repeat [3]. The technique involves mainly free associative talking and the interpretation of dreams during analytic sessions, among other processes.

We have proposed a functional and computational model which describes neurotic behavior as an associative memory process [4, 5], where the network returns a stored pattern when it is shown another input pattern sufficiently similar to the stored one [6]. We modelled the compulsion to repeat neurotic symptoms [1], by supposing that such a symptom is acted when the subject is presented with a stimulus which resembles a repressed or traumatic memory trace. The stimulus causes a stabilization of the neural net onto a minimal energy state, corresponding to the trace that synthesizes the original repressed experience, which in turn generates a neurotic response (an act). The neurotic act is not a result of the stimulus as a new situation but a response to the repressed memory. We mapped the symbolic associative process involved in psychoanalytic working-through onto a corresponding process of reinforcing synapses among memory traces in the brain.

In our model, memory functioning was originally modelled by a Boltzmann machine (BM). However, the power-law and generalized q-exponential behavior we have found, for the node-degree distributions of the network topologies generated by our model, indicate that they may not be well described by Boltzmann-Gibbs (BG) statistical mechanics, but rather by nonextensive statistical mechanics [7-9]. We have therefore modelled memory by a generalization of the BM called Generalized Simulated Annealing (GSA) [8]. In GSA, the probability distribution of the system's microscopic configurations is not the BG distribution, assumed in the BM, and this should affect the chain of associations of ideas which we are modelling.

In this paper, we review in Section 2 our associative memory model for neurosis and the self-organizing mechanism for generating hierarchically clustered memory modules, representing sensorial and symbolic memories. In Section 3, we discuss the use of GSA as a memory access mechanism, as compared to the BM. We then show results from computer simulations with some properties of these complex networks' memory access mechanisms. In the last section, we draw some conclusions and perspectives for future work.

2. ASSOCIATIVE MEMORY MODEL FOR NEUROSIS

We proposed a memory organization, where neurons be long to two hierarchically structured modules corresponding to sensorial and symbolic memories. Traces stored in sensorial memory represent mental images of stimuli received by sensory receptors. Symbolic memory stores higher level representations of traces in sensorial memory, i.e. symbols, and represents brain structures associated with symbolic processing, language and consciousness. Sensorial and symbolic memories interact, producing unconscious and conscious mental activity [4].

Memory functioning was initially modelled by a BM with N neurons, where node states take binary values and connections have symmetrical weights w_ij = w_ji. The states S_i of the units take binary output values in {0,1}. Because of the symmetry of the connections, there is an energy functional

which allows us to define the BG distribution function for network states

where T is the network temperature parameter. In the BM, pattern retrieval on the net is achieved by a standard simulated annealing process, in which the network temperature T is gradually lowered by a factor α, according to the BG distribution given by Eq. 2. A detailed treatment of the BM may be found in [6, 10].

Brain neural topology is structured by cooperative and competitive mechanisms, controlled by neurosubstances, where neurons interact mainly with spatially close neighbors, having fewer long-range synaptic connections to more distant neurons [11, 12]. This is started and controlled by environmental stimulation and is the process whereby the environment represents itself in the brain.

We summarize the clustering algorithm developed based on these biological mechanisms [4] to model the selforganizing process which results in a structured, clustered topology of each memory as follows. In Step 1, neurons are uniformly distributed in a bi-dimensional square sheet. In Step 2, we assume a Gaussian approximation for the numerical solution of the diffusion equation of neural substances that control neural competition and cooperation. A synapse is thus allocated to connect two neurons n_i and n_j, according to a Gaussian probability distribution P_ij of the distance that separates the pair, with standard deviation σ. A synapse connecting n_i to n_j has strength proportional to P_ij and weights are symmetrical. Step 3 clusterizes neurons in the memory sheets, based on mechanisms for forming cortical maps [4, 6, 13], where a group of neurons spatially close to each other represents a sensorial stimulus or an idea. Step 4 regulates synaptic intensities by strengthening synapses within a cluster and reducing synaptic strength between clusters, disconnecting them. Neurons, which have received stronger sensorial stimulation and are more strongly connected, stimulate their neighborhoods and promote even stronger connections.

Long-range synaptic growth is energetically more costly than short-range, and consequently the former is less frequent in the brain than the latter. We represent association of ideas or symbols (such as in culture and language) by long-range synapses, which should connect clusters by considering the basic Hebbian learning mechanism [6, 11, 12], where synaptic growth between two neurons is promoted by simultaneous stimulation of the pair. Since we are still unaware of synaptic distributions which result in such topologies, as a first approximation, we allocated synapses randomly among clusters. A full description of the algorithm, with memory storage and retrieval and working-through simulation, can be found in [4].

3. NETWORK TOPOLOGY AND ASSOCIATIVITY

We have generated 10000 different initial topologies with the clustering algorithm, for different values of N, such that N_sens = N_symb = N/2 of them belong to sensorial and symbolic memories respectively and σ = 0.58. In [4], we explain how we represent and interpret these as neurotic topologies, by linking neurons between sensorial and symbolic memories more weakly. Other model parameters are also described in [4]. We then measured the average node degree (k) distributions of these topologies.

Figure 1 [14] shows an asymptotic power law behavior with exponent γ ≈ -3.2, which indicates scale independence [9]. For N = 4000, the deviation from the fit by the Poisson distribution, P_λ(k) = λ^k exp(-λ)/k!, for k 10 is quite evident. The deviation from Poisson for higher values of k may be attributed to the cooperative-competitive biological mechanisms mentioned earlier, which introduce structure. Smaller values of k correspond to neurons that did not significantly participate in the competition-cooperation process and, hence, distributions for small k are approximately fitted by Poisson forms.

where p₀, δ, τ, µ and q are additional adjustable parameters. The curves indicate that, asymptotically, the power-law and generalized q-exponential fits are appropriate, with q ≈ 1.113. This is a common feature of many biological systems and indicates that they may not be well described by BG statistical mechanics.

There is no theoretical indication of the exact relation between network topology and memory dynamics. There have been some indications that complex systems which present a power-law behavior, i.e. which are asymptotically scale invariant, may be better described by the Nonextensive Statistical Mechanics formalism [8, 9, 15, 16]. Since the neural systems we are studying do not have only local interactions and present the scale-free topology characteristic, we have begun to investigate memory dynamics with a generalized acceptance probability distribution function [8] for a transition from state S ≡ {S_i} to S', if H(S') > H(S), given by

where q_A is a model parameter and other variables and parameters are the same as defined in Section 2.

The memory access mechanism is thus modelled by a generalization of the BM, called Generalized Simulated Annealing (GSA) [8], derived from the nonextensive formalism, based on the generalized acceptance probability, given by Eq. 4. In GSA, the probability distribution of the system's microscopic configurations is not the BG distribution given by Eq. 2, assumed in the BM, and this should affect the chain of associations of ideas which we are modelling. In the BM, depending on temperature, there is a tendency of reaching one of the energy minima closer to the initial configuration presented to the network. The probability distribution given by Eq. 4 should allow reaching more distant minima of the energy functional H, from an initial configuration, than the BM for the same temperature values, which corresponds to more associativity among memory traces. This implies that the GSA machine will tend to make many local associations and, more often than the BM, will also make looser, more distant associations. This would correspond to a more flexible and creative memory dynamics in the brain.

To illustrate this, we compare the energies of the patterns accessed by the BM and GSA for an initial temperature T = 0.2. Since we are searching for local minima, we use lower initial temperature values and higher values of the annealing schedule α. In the following simulations, we have analyzed smaller networks with N = 32, N_sens = N_symb = 16, since simulation of memory access is very time consuming. Memory sheets have size 1.5 X 1.5 and σ = 0.58. The simulation experiment followed was to perform up to 10000 minimization procedures, each one starting from a different random network configuration. When a new pattern is found, it is stored and the procedure is repeated from other random starting configurations, otherwise the search stops. We note in Fig. 2 that, for T = 0.2, there are many patterns found by GSA that are not found using the BM. In these simulations, GSA appears to visit state space more loosely, while the traditional BM visits state space more uniformly. For very low temperatures, the BM functions more like a gradient descent method and, when presented with randomly generated patterns, the network will stabilize at the closest local minima.

We compare, in Fig. 3 the frequency with which different patterns, corresponding to minimum energy states, are found with the BM and with the GSA for q_A = 1.4, for the same simulations of Fig. 2. We observe there is degeneracy in network states. We notice that, in the case of GSA, the frequency with which the hardest to detect patterns are found is much larger than the corresponding ones in the BM. In particular, many patterns that are not found by the BM are detected employing GSA. This corresponds to the gaps encountered in the spectrum shown in Fig. 2(a). GSA tends to prefer the lower energy states, but will also find, with low probability, higher energy states. One can observe an exponential upper limit for the frequency of visits, as a function of energy for GSA. The BM tends to visit states with a more uniform distribution of frequencies, as is expected from the characteristic of the locality of visits of state space, which we mentioned before.

4. CONCLUSIONS

We have proposed a memory organization, where two hierarchically structured modules corresponding to sensorial and symbolic memories interact, producing sensorial and symbolic activity, representing unconscious and conscious mental processes. This memory structure and functioning, along with an adaptive learning process, is used to explain a possible mechanism for neurotic behavior and psychoanalytical working-through.

The complex network topologies which result from the self-organizing processes, based on biological mechanisms, which we have proposed, have properties that are common features of many biological systems and indicate that they may not be well described by BG statistical mechanics, but rather by Nonextensive Statistical Mechanics. These mechanisms are very characteristic of much of the brain's functioning and suggest the use of a GSA algorithm to model memory functioning and the way we associate ideas in thought. The study of network quantities such as node degree distributions and clustering coefficients may indicate possible experiments, which would validate models such as the one presented here.

We are proceeding in further model refinement and analysis. It is necessary to verify more thoroughly the dependence of model behavior on its various parameters such as T and σ. These parameters represent, at least partially, the effects of neurosubstances such as neural growth factors, neurotransmitters and neuromodulators. Although we do not have experimental indications of their absolute values, the tuning of their relative values is fundamental for model stability and functioning. This may give some insight to basic mechanisms in real neural networks and the emergence of behavioral aspects. We are also interested in trying to map language structure and processing into network topology and dynamics, although we are not yet sure whether this is possible.

Our main contribution in recent work has been to propose a neuronal model, based on known microscopical, biological brain mechanisms, which describes conscious and unconscious memory activity involved in neurotic behavior, as described by Freud. The model emphasizes that symbolic processing, language and meaning are important for consciousness. However, we do not sustain or prove that this is the actual mechanism that occurs in the human brain. Although biologically plausible, in accordance with many aspects described by psychodynamic and psychoanalytic clinical experience, and experimentally based on simulations, the model is very schematic. It nevertheless seems to be a good metaphorical view of facets of mental phenomena, for which we seek a neuronal substratum, and suggests directions of search.

Acknowledgements

We are grateful for fruitful discussions regarding basic concepts of psychoanalytic theory, with psychoanalysts and professors Romildo do Rêgo Barros, Paulo Vidal and Angela Bernardes of the Escola Brasileira de Psicanálise of Rio de Janeiro and the Department of Psychology of the Universidade Federal Fluminense. We also thank the members of the Escola Brasileira de Psicanálise of Rio de Janeiro for their warm reception of the author R. S. Wedemann in some of their seminars. Suggestions and discussions with Sumiyoshi Abe, Evaldo M. F. Curado and Constantino Tsallis regarding complex networks and nonextensive statistical mechanics were also helpful and enlightening. We acknowledge the Brazilian National Research Council (CNPq) and the Rio de Janeiro State Research Foundation (FAPERJ) for partial financial support.

(Received on 26 January, 2009)

Publication Dates

History