Elsevier

Journal of Theoretical Biology

Volume 336, 7 November 2013, Pages 1-10
Journal of Theoretical Biology

A critical study of network models for neural networks and their dynamics

https://doi.org/10.1016/j.jtbi.2013.07.005Get rights and content

Abstract

We use three network models, Erdős–Rényi, Watts–Strogatz and structured nodes, to generate networks sharing several topological features with the neural network of C. elegans (our target network). A new topological measurement, incoming and outgoing edges heat maps, is introduced and used to compare the considered networks. We run these networks as random recurrent neural networks and study their dynamics.

We find out that none of the considered network models generates networks similar to the target one both in their topological features and dynamics. Moreover, we find that the dynamics of the target network are very robust to the rewiring of its edges.

Introduction

In the previous years several fields of Biology saw important advances in the study of processes and phenomena when they are regarded as networks (Junker and Schreiber, 2008, Kleinberg and Easley, 2011). This study has been fuelled by the understanding of links between topology and dynamics (Milo et al., 2002, Milo et al., 2004, Alon, 2006) and by the definition of network models that are able to generate networks with topological features present in biological networks (Barabàsi and Albert, 1999, Chung et al., 2003, Frisco, 2011).

A similar development took place also in neural networks. Recent studies linked neural network topology and dynamics in a novel way (Bock et al., 2011, Ko et al., 2011, Perin et al., 2011, Sporns et al., 2000, Bassett and Bullmore, 2006, Lu et al., 2009, Lago-Fernández et al., 2000). These important discoveries have not been matched by new network models that are able to closely replicate neural networks. Researches using network models in order to generate networks similar to neural networks employed either classical network models (i.e., Erdös–Rényi) or more recent ones (e.g., Watts–Strogatz) that are able to replicate only some (often just one) topological features present in the neural networks.

The ability to generate networks similar to empirical neural networks is paramount: it allows us to test theories that cannot be tested in empirical networks, it allows us to understand how the network could behave under different stimuli, etc. Moreover, one should know that simulating the dynamics of a network is a costly undertaking and, even if very simplified network models, it is not possible to simulate in real-time the dynamics of a network with more than a few tens of thousands nodes (Izhikevich, 2003, Vogels et al., 2005), and an exhaustive exploration of the state space of a network with more than a few hundreds nodes is not feasible (Drossel, 2008). It would be helpful if the dynamics of a network could be inferred from the topological features of the network without any simulation on the dynamics. This would allow faster analysis of network dynamics and the ability to infer the dynamics of very large networks.

In this paper we take a critical look at the network models used to replicate neural networks. Some of the questions we address are as follows: How similar are the topologies of the networks generated by the network models to the ones of the empirical neural networks that they aim to replicate? Are there, for neural networks, topological properties describing these networks better than the others? Are the dynamics of empirical and generated networks comparable?

As a case study we consider the neural network of C. elegans. In the following we refer to this network as the target network. We chose this network because it is relatively small, its topology is completely known and it has been extensively used as a benchmark for several studies (Watts and Strogatz, 1998, Milo et al., 2002). We use the connected component of 297 neurons with the synaptic links between them. We included both chemical and electrical synapses (gap junctions) in the network and treat them equally (Majewska and Yuste, 2001). We aggregated multiple connections from one node to another as a single edge in the network, for this reason we cannot differentiate between different connection types. We computed several topological features of this neural network and we tested its dynamics under different stimuli when it is regarded as a random recurrent neural network. Then, using known network models, we generated networks having topological features as similar as possible to our case study, we treated these networks as random recurrent neural networks, and we tested their dynamics under different stimuli.

We found out that there is very little relation between similarities in dynamics and similarities in topology between our case study and the generated networks. This means that similar dynamics between a particular generated network and the target network do not imply similar topology. Also the opposite does not hold true: artificial networks having topological features similar to the target network do not share similar dynamics.

The present paper poses more questions than it actually answers (see Section 6). Overall, our findings can be summarised saying that none of the current network models seems to be appropriate to replicate the target network. If one wants to have networks similar (both in topology and dynamics) to the target network, then it is better to obtain other networks simply by perturbing (i.e., applying a filter/noise) the edges of the target network. Put in different terms, this research lets us realise even more the pitfalls in which it is possible to incur when trying to replicate complex networks. This proves that the classical network models considered by us are not appropriate to model neural networks. As a consequence, we conclude that other network models (possibly including other elements as development, topography, etc.) should be pursued to replicate neural networks.

The rest of the paper is organised as follows. The initial sections give a background on networks and network topological properties we considered (Section 2), the network models we used (Section 3) and the model of neural networks we adopted (Section 4). The followed methodology and obtained results are described in Section 5. In Section 6 we give our remarks on our study. The appendices give further details on one of the considered network models.

Section snippets

Networks and their topological properties

In this section we introduce the network terminology that we employ together with short definitions of the topological features we considered. Further details on network topological features can be found in Junker and Schreiber (2008).

Networks are composed of nodes connected by edges. We consider directed connected networks. This means that any edge can be traversed in one way but not in the other and if edges were considered bi-directional, then there would be a path from any node to any other

Network models

We generated networks using three network models briefly outlined in the following. Each of these three models constructs networks in a different fashion and the networks obtained by these models have very different topological properties.

The Erdős–Rényi (ER) model (Erdös and Rényi, 1959) (the networks generated by it are also known as random networks) starts from a fixed number of nodes and it adds edges with a fixed probability pER. Networks created by the ER model are likely to have a low

Random recurrent neural networks and their dynamics

As indicated in Section 1 we studied the dynamics of the considered networks treating them as random recurrent neural networks (RRNNs). This is a common model of neural networks (Dauce et al., 1998, Siri et al., 2006, Sompolinsky et al., 1988). These networks use point neurons (neurons whose activity is represented by a single value modelling electrical potential difference rather than modelling it over several values as in a multi-compartment neuron) which allow for simulations to be performed

Generated networks

We generated 10 networks for each of the three considered network models. The generated networks were intended to have topological measurements similar to those of the target network.

The ER model was used to generate 10 networks with the probability pER=0.026. This value was used to generate networks with an average degree as close as possible to one of the target networks. This value for the probability was calculated by p=desiredavg.degree/(nodesnodes1), which gives a probability that can

Final remarks

As we said in the Introduction, this paper poses more questions than it actually answers. Here we explicitly pose some of these questions.

We believe that, when comparing complex networks, several topological features have to be considered. This is simply because a few such features cannot encapsulate the complexity present in these networks. The small-world property is definitely not sufficient to compare networks: it does not say anything, for instance, about the degree distribution and other

References (35)

  • N. Brunel

    Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons

    Journal of Computational Neuroscience

    (2000)
  • F. Chung et al.

    Duplication models for biological networks

    Journal of Computational Biology

    (2003)
  • A. Clauset et al.

    Power-law distributions in empirical data

    SIAM Review

    (2009)
  • P. Dayan et al.

    Theoretical NeuroscienceComputational and Mathematical Modeling of Neural Systems

    (2001)
  • B. Drossel
    (2008)
  • P. Erdös et al.

    On random graphs

    Publicationes Mathematicae

    (1959)
  • P. Frisco

    Network model with structured nodes

    Physical Review E

    (2011)
  • Cited by (2)

    • Relating network connectivity to dynamics: opportunities and challenges for theoretical neuroscience

      2019, Current Opinion in Neurobiology
      Citation Excerpt :

      In [42] it was also found that characteristic path length and average betweenness centrality alone do not predict the emergence of population synchronization of bursting neurons, and only when degree distribution is also taken into account can this property be understood. Finally, Govan et al. [28] considered dynamics similar to those of Boolean networks on the C. elegans connectome and compared the network activity to that of Erdös–Rényi random and Watts–Strogatz small world networks that were matched according to average degree, average shortest path length and clustering coefficient. They found dramatically different likelihoods of regular network oscillations across these networks, despite matching parameters, indicating the lack of dynamic relevance of these measures.

    • Simulation-based sensor allocation for dynamic environment estimation in cyber-physical building system

      2019, IEEE International Conference on Automation Science and Engineering
    View full text