A critical study of network models for neural networks and their dynamics
Introduction
In the previous years several fields of Biology saw important advances in the study of processes and phenomena when they are regarded as networks (Junker and Schreiber, 2008, Kleinberg and Easley, 2011). This study has been fuelled by the understanding of links between topology and dynamics (Milo et al., 2002, Milo et al., 2004, Alon, 2006) and by the definition of network models that are able to generate networks with topological features present in biological networks (Barabàsi and Albert, 1999, Chung et al., 2003, Frisco, 2011).
A similar development took place also in neural networks. Recent studies linked neural network topology and dynamics in a novel way (Bock et al., 2011, Ko et al., 2011, Perin et al., 2011, Sporns et al., 2000, Bassett and Bullmore, 2006, Lu et al., 2009, Lago-Fernández et al., 2000). These important discoveries have not been matched by new network models that are able to closely replicate neural networks. Researches using network models in order to generate networks similar to neural networks employed either classical network models (i.e., Erdös–Rényi) or more recent ones (e.g., Watts–Strogatz) that are able to replicate only some (often just one) topological features present in the neural networks.
The ability to generate networks similar to empirical neural networks is paramount: it allows us to test theories that cannot be tested in empirical networks, it allows us to understand how the network could behave under different stimuli, etc. Moreover, one should know that simulating the dynamics of a network is a costly undertaking and, even if very simplified network models, it is not possible to simulate in real-time the dynamics of a network with more than a few tens of thousands nodes (Izhikevich, 2003, Vogels et al., 2005), and an exhaustive exploration of the state space of a network with more than a few hundreds nodes is not feasible (Drossel, 2008). It would be helpful if the dynamics of a network could be inferred from the topological features of the network without any simulation on the dynamics. This would allow faster analysis of network dynamics and the ability to infer the dynamics of very large networks.
In this paper we take a critical look at the network models used to replicate neural networks. Some of the questions we address are as follows: How similar are the topologies of the networks generated by the network models to the ones of the empirical neural networks that they aim to replicate? Are there, for neural networks, topological properties describing these networks better than the others? Are the dynamics of empirical and generated networks comparable?
As a case study we consider the neural network of C. elegans. In the following we refer to this network as the target network. We chose this network because it is relatively small, its topology is completely known and it has been extensively used as a benchmark for several studies (Watts and Strogatz, 1998, Milo et al., 2002). We use the connected component of 297 neurons with the synaptic links between them. We included both chemical and electrical synapses (gap junctions) in the network and treat them equally (Majewska and Yuste, 2001). We aggregated multiple connections from one node to another as a single edge in the network, for this reason we cannot differentiate between different connection types. We computed several topological features of this neural network and we tested its dynamics under different stimuli when it is regarded as a random recurrent neural network. Then, using known network models, we generated networks having topological features as similar as possible to our case study, we treated these networks as random recurrent neural networks, and we tested their dynamics under different stimuli.
We found out that there is very little relation between similarities in dynamics and similarities in topology between our case study and the generated networks. This means that similar dynamics between a particular generated network and the target network do not imply similar topology. Also the opposite does not hold true: artificial networks having topological features similar to the target network do not share similar dynamics.
The present paper poses more questions than it actually answers (see Section 6). Overall, our findings can be summarised saying that none of the current network models seems to be appropriate to replicate the target network. If one wants to have networks similar (both in topology and dynamics) to the target network, then it is better to obtain other networks simply by perturbing (i.e., applying a filter/noise) the edges of the target network. Put in different terms, this research lets us realise even more the pitfalls in which it is possible to incur when trying to replicate complex networks. This proves that the classical network models considered by us are not appropriate to model neural networks. As a consequence, we conclude that other network models (possibly including other elements as development, topography, etc.) should be pursued to replicate neural networks.
The rest of the paper is organised as follows. The initial sections give a background on networks and network topological properties we considered (Section 2), the network models we used (Section 3) and the model of neural networks we adopted (Section 4). The followed methodology and obtained results are described in Section 5. In Section 6 we give our remarks on our study. The appendices give further details on one of the considered network models.
Section snippets
Networks and their topological properties
In this section we introduce the network terminology that we employ together with short definitions of the topological features we considered. Further details on network topological features can be found in Junker and Schreiber (2008).
Networks are composed of nodes connected by edges. We consider directed connected networks. This means that any edge can be traversed in one way but not in the other and if edges were considered bi-directional, then there would be a path from any node to any other
Network models
We generated networks using three network models briefly outlined in the following. Each of these three models constructs networks in a different fashion and the networks obtained by these models have very different topological properties.
The Erdős–Rényi (ER) model (Erdös and Rényi, 1959) (the networks generated by it are also known as random networks) starts from a fixed number of nodes and it adds edges with a fixed probability pER. Networks created by the ER model are likely to have a low
Random recurrent neural networks and their dynamics
As indicated in Section 1 we studied the dynamics of the considered networks treating them as random recurrent neural networks (RRNNs). This is a common model of neural networks (Dauce et al., 1998, Siri et al., 2006, Sompolinsky et al., 1988). These networks use point neurons (neurons whose activity is represented by a single value modelling electrical potential difference rather than modelling it over several values as in a multi-compartment neuron) which allow for simulations to be performed
Generated networks
We generated 10 networks for each of the three considered network models. The generated networks were intended to have topological measurements similar to those of the target network.
The ER model was used to generate 10 networks with the probability . This value was used to generate networks with an average degree as close as possible to one of the target networks. This value for the probability was calculated by , which gives a probability that can
Final remarks
As we said in the Introduction, this paper poses more questions than it actually answers. Here we explicitly pose some of these questions.
We believe that, when comparing complex networks, several topological features have to be considered. This is simply because a few such features cannot encapsulate the complexity present in these networks. The small-world property is definitely not sufficient to compare networks: it does not say anything, for instance, about the degree distribution and other
References (35)
- et al.
Self-organization and dynamics reduction in recurrent networksstimulus presentation and learning
Neural Networks
(1998) - et al.
Topological target patterns and population oscillations in a network with random gap junctional coupling
Neurocomputing
(2001) - et al.
Impact of network topology on decision-making
Neural Networks
(2009) - et al.
Topology of gap junction networks in C. elegans
Journal of Theoretical Biology
(2001) An Introduction to Systems Biology
(2006)- et al.
Emergence of scaling in random networks
Science
(1999) - et al.
Small-world brain networks
The Neuroscientist
(2006) - et al.
A duplication growth model of gene expression networks
Bioinformatics
(2002) - et al.
Network anatomy and in vivo physiology of visual cortical neurons
Nature
(2011) Boolean network models of cellular regulationprospects and limitations
Journal of the Royal Society Interface
(2008)
Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons
Journal of Computational Neuroscience
Duplication models for biological networks
Journal of Computational Biology
Power-law distributions in empirical data
SIAM Review
Theoretical NeuroscienceComputational and Mathematical Modeling of Neural Systems
On random graphs
Publicationes Mathematicae
Network model with structured nodes
Physical Review E
Cited by (2)
Relating network connectivity to dynamics: opportunities and challenges for theoretical neuroscience
2019, Current Opinion in NeurobiologyCitation Excerpt :In [42] it was also found that characteristic path length and average betweenness centrality alone do not predict the emergence of population synchronization of bursting neurons, and only when degree distribution is also taken into account can this property be understood. Finally, Govan et al. [28] considered dynamics similar to those of Boolean networks on the C. elegans connectome and compared the network activity to that of Erdös–Rényi random and Watts–Strogatz small world networks that were matched according to average degree, average shortest path length and clustering coefficient. They found dramatically different likelihoods of regular network oscillations across these networks, despite matching parameters, indicating the lack of dynamic relevance of these measures.
Simulation-based sensor allocation for dynamic environment estimation in cyber-physical building system
2019, IEEE International Conference on Automation Science and Engineering