Generative framework for dimensionality reduction of large scale network of non-linear dynamical systems driven by external input

Several studies have proposed constraints under which a low dimensional representation can be derived from large-scale real-world networks exhibiting complex nonlinear dynamics. Typically, these representations are formulated under certain assumptions, such as when solutions converge to attractor states using linear stability analysis or using projections of large-scale dynamical data into a set of lower dimensional modes that are selected heuristically. Here, we propose a generative framework for selection of lower dimensional modes onto which the entire network dynamics can be projected based on the symmetry of the input distribution for a large-scale network driven by external inputs, thus relaxing the heuristic selection of modes made in the earlier reduction approaches. The proposed mode reduction technique is tractable analytically and applied to different kinds of real-world large-scale network scenarios with nodes comprising of a) Van der Pol oscillators b) Hindmarsh-Rose neurons. These two demonstrations elucidate how order parameter is conserved at original and reduced descriptions thus validating our proposition.

Large-scale dynamical systems are useful tools to explain a wide variety of complex phenomena in nature e.g. financial markets [1], jamming transitions [2], human mobility dynamics [3], weather patterns [4] and brain dynamics [5]. While increase in scale or dimensions may increase the predictive power of the model system, nonetheless a reduction to simpler descriptions at lower dimensions is critical for having relevance to empirical observations and analytical tractability of underlying mechanisms governing empirical observations. One robust approach of reducing dimensions is defining modes on which the original system can be projected [4,6]. The selection of a mode is often heuristically motivated, and the mode can also be an order parmeter from the perspective of the paradigmatic framework of Synergetics [7]. In Neuroscience, reduction of dynamical systems with respect to modes constructed from distribution of external input has been performed earlier on small-scale network of linearly coupled excitable systems [6]. Since this reduction retain important network dynamics, large-scale networks were conceptualized by coupling these reduced systems with long-range coupling [8,9], the later being heuristically argued from symmetry properties. In present work, we perform reduction on a large-scale network where connection among nodes involve global and local coupling mimicking a real-world system. Subsequently, long-range coupling term between modes in the reduced system is derived analytically as part of the reduction process. Global coherence is an order parameter that can be computed both at the level of original dynamical system as well as from the mode dynamics in the reduced system. Conservation of global coherence at both levels is used to validate the generality of our approach in two distinct networks. First, we simulate a large-scale network where each node is a Van der Pol oscilator having 2-dimensional dynamics and coupled us-ing local and global parameters. Second each node is a Hindmarsh-Rose neuron, a 3-dimensional dynamical system which can exhibit different time scales of oscillations resulting in bursting along with tonic spiking behavior.
Each i th excitable system in node n is represented by a vector of state variables (state vector) x (n) i . Input to i th excitable system from interaction with other excitable systems within n th node is given by a vector function g {x   i ) contributes to the local dynamics of i th excitable system in n th node. Then, the dynamics of the entire system is described by the following set of equations whereẋ is time derivative of state vectors, φ is a function of the external input (I) and τ n is the time-constant of n th node which is a differentiating factor between nodes. However, within n th node, external current to i th excitable system (I where I is a continuous variable having normal distribution N(µ, σ). Now, we can represent x(t, I) as a superposition of M bi-orthogonal modes {v i } where R(t, I) is the residual and M ≪ J ( Figure 1B). The nature of reduction is such that dynamical system given in (1) is reduced to solving for the mode coefficients ξ i as described in the following set of equations where and {v + i } are the adjoint basis for the biorthogonal modes {v i }.

Large-scale network of Van der Pol oscillators:
A Van der Pol oscillator [10] has two state variables x and y which follows the following equationṡ A large-scale network where individual node is essentially a Van der Pol oscillator can be cast into equation (1) with the following relations where a is a constant and E[{x j }] is the expected value of the state-variables {x j }. For the reduced system described in (4) with Large-scale network architecture of complete system where input to a node is the weighted sum of expected value of state-variables B) Large-scale network architecture of proposed reduced system where input to a target-mode is weighted sum of target-mode specific expected mode activity from other nodes.
where the constants a i and A ij are computed by applying bi-orthogonal assumption of modes as stated in Appendix A.
Large-scale network of Hindmarsh-Rose neurons: In Neuroscience, Hindmarsh-Rose (HMR) neuron is a three dimensional model of single neuron firing dynamics having three state variables [11] x =y − ax 3 where a, b, c, d, r, s and x0 are the constants. A network of excitatory and inhibitory HMR neurons have been used to describe a small-scale network of neurons [6]. Thus, a node in the brain (Fig 1a) can be expressed as a six dimensional state space with 3 excitatory and 3 inhibitory variables represented as x = [x 1 , y 1 , z 1 , x 2 , y 2 , z 2 ] T . Extending this architecture, for the large-scale system in (1), we obtain where K 11 , K 12 and K 21 are coupling constants between excitatory and inhibitory nodes. For the reduced system in (4), ξ i = [α 1 , β 1 , γ 1 , α 2 , β 2 , γ 2 ] T , we derive We simulated a network of three nodes with each functional unit in a node is governed by 1) Van der Pol (VDP) oscillator or 2) HMR neurons. We select the node connection matrix of the given form where w is a scalar in the range 0 to 1 representing negligible node interaction to strong node interaction. We also considered different values of K for VDP (and Inhibitionexcitation ratio, IER = K12 K11 for HMR) and for different values of mean current µ. For validating reduced system with M modes, we compare the Global Coherence (GC) evaluated using the complete system (C p ) with GC evaluated using the reduced system (R p ) using the following where f is frequency and T is total number of pairs < f, w, K, µ >. GC is computed between mean activities of each node. For the reduced system, the mean activity of each node is the mean of individual excitable systems' activities estimated using (3) without the residual.
As M increases, error in proposed reduction process decreases more rapidly as compared to previous approaches (Figure 2A & B) for both large-scale networks using Van der-Pol and Hindmarsh-Rose models as nodes, thus, validating our reduction approach. For Van der-Pol model, reduction proposed in [9] generated numerical instability during simulation. Global coherence calculated from proposed reduction for an exemplar parameter space matches closely with the original system unlike heuristic approaches ( Figure 2C). We further validate our framework by showing the reproduction of time-series of mean-field activity of each HMR and VDP node ( Figure  3) for several different parameters spaces.
In [6], local-interaction between state-variables was facilitated via the mean field activity of the node which by itself is a small-scale network. In the reduced model this interaction was preserved as the input to state variable of mode i was characterized via mode-specific output of the node ( M j=1 A ij α j ) governed by matrix A which is obtained as a part of reduction. However, in the case of the large-scale network of these reduced nodes the input to state-variable of mode i of node m was either the activity of state-variable of mode i from other nodes [8] or it was the sum of activities of state-variables of all modes from other nodes [9]. In this paper, the input to mode i of node m is derived by projecting the long-range inter-action term of complete large-scale network to modes of the external input. Thus, our proposed scheme preserves the original long range interaction (Figure 2 and Figure  3).
In summary, we propose a generalized scheme for reduction of the dynamics of a large-scale network into lower dimensional mode description based on properties of the external input. Obviously, any such reduction lowers the computational complexity. However an important point to note is a model's benefit is not necessarily limited to mimicking the complex dynamics of real-world system. For example, a detailed model of the cortical layer will be highly informative [12], but not necessarily insightful for explaining the cortical interactions during a specific behavioral task. How do we develop taskspecific insights from a high dimensional data with minimal assumptions will be critical question for future studies and how they could profit from our proposed formalism. Our approach is best-placed to application where task inevitably involves reduction from high-dimensional state space to functionally well defined lower dimensional modes. Such problems are also not necessarily exclusively brain specific, but also pertinent for climate dynamics, traffic problems, or anywhere dynamical systems are driven by external time varying and distributed input and the approach presented in this article may be insightful for a wide range of scientific disciplines.
i (I)dI g1(I) and g2(I) are the pdfs of external input to excitatory and inhibitory sub-populations respectively.