Effect of biologically-motivated energy constraints on liquid state machine dynamics and classification performance

Equipping edge devices with intelligent behavior opens up new possibilities for automating the decision making in extreme size, weight, and power-constrained application domains. To this end, several recent lines of research are aimed at the design of artificial intelligence hardware accelerators that have significantly reduced footprint and power demands compared to conventional CPU/GPU systems. However, despite some key advancements, the majority of work in this area assumes that there is an unlimited supply of energy available for computation, which is not realistic in the case of battery-powered and energy harvesting devices. In this paper, we address this gap by exploring the computational effects of energy constraints on a popular class of brain-inspired spiking neural networks–liquid state machines (LSMs). Energy constraints were applied by limiting the spiking activity in subsets of LSM neurons. We tested our designs on two biosignal processing tasks: epileptic seizure detection and biometric gait identification. For both tasks, we show that energy constraints can significantly improve classification accuracy. This demonstrates that in the design of neuromorphic systems, reducing energy and increasing performance are not always competing goals.


Introduction
Enhancing automated decision making capabilities of internet-of-things devices is a key step towards harnessing the true power of artificial intelligence (AI) at the edge. However, one of the key challenges in this endeavor is that edge devices (sensors, mobile phones, autonomous vehicles, etc) typically have significant constraints that limit their size, weight, and power (SWaP), as well as their cloud connectivity. Moreover, several applications may not be able to afford the extra latency associated with offloading computations to the cloud, even if they do have enough energy available to transmit all of the data [1]. Therefore, in a number of cases, it is necessary to implement AI algorithms directly on the edge devices. This requires innovation in both hardware and algorithm design in order to meet devices' SWaP constraints while maintaining acceptable levels of performance (e.g. classification accuracy). On the hardware design side, neuromorphic chips like IBM TrueNorth [2], Intel Loihi [3], and several others [4], are paving the way for low-power acceleration of bio-inspired spiking neural networks (SNNs) and state-of-the-art deep neural networks. On the algorithm side, techniques are being explored to compress neural networks so that they can be more efficiently implemented on resourceconstrained hardware [5]. What's interesting is that, in the design of these AI systems, hardware concepts like energy and algorithm concepts like classification accuracy tend to be decoupled so that one has little direct effect on the other. We note that this is in stark contrast to biological systems, where energy efficiency and intelligent behavior were co-optimized through evolution and are intimately entangled. As a result, biology is finely-tuned to make the best use of the energy that an organism has available as it makes and executes decisions. Ultimately, we would like to achieve this same level of energy awareness in artificial systems.
Energy plays a number of important roles in biological systems. For example, energy constraints in the human brain serve to maintain stability of neuronal circuits. In [6], the authors suggest that disruptions in brain energy metabolism contributes to the generation and maintenance of epileptic seizures. In the human brain, energy metabolism is closely regulated by glial cells, the most common being astrocytes. Astrocytes attach themselves to blood vessels in the brain and break down glucose into metabolites that get distributed to neurons, which can consume them for energy [7]. Astrocytes also maintain a store of glycogen, which can serve as a reserve energy supply for neurons [6]. In this paper, the energy-regulating role of astrocytes is abstracted to explore the effects of a dynamic energy supply on SNN dynamics and performance. Other works have recently investigated the impact of energy constraints on artificial neural networks (ANNs). These include analyzing the impact of simple energy constraints on artificial spiking neurons [8], creating a bio-realistic neuron-glial model for energy constraints [9], analyzing the impact of adding artificial astrocytes to a neural network [10], and training a neural network to operate within arbitrarily-defined energy constraints [11]. However, to our knowledge, no research has been done to analyze the impact of energy constraints on a neural network trained to perform a computational task. Therefore, we provide the following contributions in this work: • Design and implementation of a bio-inspired SNN-liquid state machine (LSM)-with energy constraints in the form of limited spike activity in subsets of neurons. • Analysis of the impact of energy constraints on LSM dynamics through Lyapunov exponent and separation ratio metrics. • Evaluation of energy-constrained LSM performance on two biosignal processing tasks: epileptic seizure detection and biometric gait identification. The rest of this paper is organized as follows: section 2 provides background on LSMs and gives a detailed description of related work. Section 3 discusses our energy-constrained LSM design, simulation methodology, and evaluation strategy. Section 4 presents results of the energy-constrained LSM design, and a comparison with literature is made in section 5. Section 6 provides an in-depth discussion of our results. Finally, section 7 concludes this work and provides avenues for future research.

Background and related work 2.1. Liquid state machines
LSMs represent one type of reservoir computing-a machine learning paradigm in which spatiotemporal data are mapped to a random dynamical system operating near the edge of chaos (a.k.a. the 'reservoir'). The reservoir extracts random features from the input which can be used to perform classification and regression tasks. First introduced by Maass [12], LSMs employ a randomly-connected, untrained recurrent SNN as their reservoir. The basic LSM structure with N inputs, H reservoir neurons, and M outputs is shown in figure 1. The LSM's output y(t) ∈ R M at time t is usually taken as a linear combination of the reservoir features: where x(t) ∈ R H is the state of the reservoir and W out ∈ R M×H is the output weight matrix. The state evolves over time according to the inputs and the dynamics of the reservoir. For a discrete time LSM: x(t) = L(W in u(t), x(t − 1); W res ), where u(t) ∈ R N is the input, W in ∈ R H×N is a random weight matrix connecting the input to the reservoir, W res ∈ R H×H is a random weight matrix for the recurrent connections in the reservoir, and L describes the dynamics of the reservoir, which will depend on what models are employed for the spiking neurons and synaptic connections. Given m training examples, the LSM can be trained using linear least squares: where X ∈ R m×H and Y ∈ R m×M are the matrices of reservoir states and target outputs corresponding to the m training examples, respectively. Note that training only consists of modifying W out , while the randomlyinitialized W in and W res matrices are fixed.
Other works have employed LSMs with good results for various applications. A very popular task for LSMs is speech recognition. Works such as [13] extensively analyze the performance of LSMs for speech recognition with various network parameters with generally good results. Examples of other time-series tasks LSMs have been used to solve include epileptic seizure detection from electroencephalogram (EEG) data and person identification based on gait data measured by a smartphone accelerometer, both solved with an LSM in [14].

Energy constrained artificial neural networks
The impact of energy constraints on SNNs has been explored previously. In [8], Burroni et al study the dynamics of SNNs where each neuron belongs to an 'energy pool', which slowly replenishes over time. Each time a neuron spikes, it drains a percentage of its associated energy pool. When its energy pool is depleted, the neuron is prevented from spiking. The authors evaluated the network by assigning each neuron a bias and analyzing the network activity and remaining energy over time. They found that oscillations in both the spike rate and network energy were present, similar to oscillations observed in biological brain activity. Another work by the same lab [9] developed an extension to the Izhikevich neuron to model the role of glial cells in regulating neuronal energy supply, with one glial cell per neuron. They found that networks of their modified neurons could reproduce specific types of spiking activities observed in the brain. However, we note that in biology, emulating the connection style of glial cells with neurons in the mammalian brains-according to an investigation on astrocyte connectivity within mouse brains-would require astrocytes to supply energy for more than one neuron: 4 on average with a limit of 8 [15].
Work has also been completed on training networks with energy constraints applied. In [11], the authors found that they were able to train the weights in an SNN such that the network operated within some specified energy constraints, with the neurons consuming energy both when integrating inputs as well as generating spikes. They found that they were able to do this regardless of the number of neurons or synaptic connections present in the network. In addition to work done examining the effect of energy constraints on SNNs, research has also been performed to analyze the impact of including artificial astrocytes in a feedforward ANN. In [10], the authors modeled the influence of astrocytes on synaptic efficacy based on neuron spiking activity. They found that, for the most part, artificial astrocytes improved the performance of multilayer feedforward networks. In contrast to prior work, this paper intends to evaluate the impact of energy constraints similar to those used in [8] on the operation of an LSM that is trained to perform a computational task (biosignal classification), especially when multiple neurons compete for the same energy supply.

Datasets
We chose to evaluate our energy-constrained LSM design with two biosignal classification tasks: epileptic seizure detection and biometric gait identification. Each dataset is described below.

Epileptic seizure detection
The epileptic seizure detection dataset was obtained through the UCI Machine Learning Repository [16,17]. This dataset contains time-series EEG data taken from patients in five different conditions, each with 100 separate samples: measurements from healthy volunteers with their eyes open, measurements when the volunteers' eyes were closed, measurements from patients suffering from epilepsy (currently no seizure) from two different brain regions, and measurements taken during an epileptic seizure. For this work, only the first and final classes were utilized and a binary classification (seizure or not) is made by the constructed LSM. Each sample of this dataset consists of roughly 23.5 s of time-series data sampled at 173.61 Hz (the speed of the amplifier used in the EEG experiment [17]) gathered from a single EEG electrode monitoring patients' brain activity. Since only two classes of the dataset were used, we had 200 samples. 70% of these 200 samples were used for training and 30% for testing. We pre-processed this dataset in two different ways. For the first type of pre-processing, as recommended by the dataset collectors, a bandpass filter from 0.53 to 40 Hz was applied [17]. Afterwards, feature scaling was done by dividing each timestep of each sample by the maximum of that sample. For the second type of pre-processing, in addition to the steps above, the data were also split into four channels by applying four bandpass filters [14]. The bandpass filter ranges used are 0-3 Hz, 4-8 Hz, 8-13 Hz, and 13-30 Hz. See figures 2 and 3 for examples of the seizure and healthy classes for each pre-processing type.

Biometric gait identification
The biometric gait identification dataset was also obtained through the UCI Machine Learning Repository [16]. It consists of accelerometer data collected from mobile phones placed in 22 different test subjects' pockets while they walked [18]. Data from this dataset is formatted as three channels (from the X, Y and Z axes) sampled at 32 Hz on average. The length of the data available varied for each user. This dataset was adapted by only considering the two users for whom there was the most data available and training the LSM to discriminate between those two users, leading to a total size of 266 samples (137 for one user and 129 for the other). As in the epileptic seizure detection task, 70% of the data were used for training and 30% were used for testing. Figure 4 depicts a sample from each user of this biometric gait identification task.

LSM topology
We used a standard LSM topology with input, reservoir, and readout layers, as described in section 2.1. All LSM hyperparameters, which are discussed in this section, are listed in table 1. Note that we have empirically found a set of baseline parameters (set A in table 1) to serve as a starting point for our simulations. In the rest of the paper, we refer to the LSM with these parameters as the 'handcrafted' LSM. The LSM input layer is sparsely connected to the reservoir layer and the reservoir layer has sparse recurrent connections. In contrast, the readout layer is fully connected to the reservoir layer. The input layer of the network can be thought of as a single neuron per input feature of the dataset. The epileptic seizure detection dataset, for example, only has one feature, so it has one input neuron. The input neurons are connected to randomly selected neurons in the reservoir with a specified probability, seen as p input in table 1. All reservoir neurons are leaky integrate-and-fire (LIF) neurons. Synaptic connections between the input neurons and the reservoir neurons are initially sampled from a uniform random distribution. The initial size of the reservoir for both datasets evaluated was set to 100 neurons.
The synaptic weights between the reservoir and the readout layer of the LSM were obtained using linear least squares, shown in (3). The readout layer is a single neuron in both tasks evaluated. A Boolean value is obtained from this readout layer by thresholding the output. The threshold value was taken to be the average LSM output over the training data. Instead of directly acting on the state x(t) of the reservoir's spiking neurons, the readout layer acts on the activity a(t) ∈ R H of the reservoir after the final timestep of the input has been applied. The reservoir activity increases each time a neuron spikes and decays during timesteps where the neuron did not spike. This method of tracking neuron spiking activity was adapted from a simplified short-term plasticity method presented in [19]: where x i (t) = 1 when reservoir neuron i spikes at time t, and is 0 otherwise. α activity and β activity are constant scalars. In [19], a represents synaptic efficacy, but we use it as a measure of reservoir activity by setting the α activity term to be a strictly negative constant. For both versions of the epileptic seizure dataset, the time-series data is not converted into spike trains to reduce the computational costs of the network. Instead, each channel of each dataset is fed as direct current input to LIF neurons in the reservoir. Because of this, the current value for a recurrent spike was selected according to the magnitude of the inputs. This was done by taking the average magnitude of the inputs over all timesteps and all samples and multiplying it by a scalar to account for the relatively low frequency of recurrent connections compared to the inputs. This scalar can be seen as α recurrent in table 1.
Neurons in the reservoir come in two varieties; either inhibitory or excitatory, the difference being the sign of their outgoing synaptic weights. Reservoir neurons have a 15% chance of being inhibitory, contrasted with the commonly used value of 20%. This difference was created after observing poor classification performance when 20% of the neurons were inhibitory; observing the LSM spiking activity, it was seen that many fewer spikes were emitted. All outbound synapses for an inhibitory neuron have a negative value, while those of an excitatory neuron will be positive. Each neuron within the reservoir is randomly connected with other neurons in the reservoir to implement recurrent connections. Intra-reservoir connections are made by (5), which was adopted from the work of [12]. The location of each neuron was chosen to be a random point in a 1 × 1 × 1 3D grid (see figure 5), with each coordinate of the neuron's location chosen by sampling from a uniform random distribution. Parameter values used in (5) can be found in table 1. Each intra-reservoir connection weight is initially set by sampling from a uniform random distribution. After all connections are  formed, synaptic scaling is performed by normalizing the sum of excitatory and the sum of inhibitory synaptic weights to Σ E and Σ I , respectively. These normalization constants along with the other parameters used when evaluating the LSM can be found tabulated in table 1.
In (5), p i,j is the probability of connection between neuron i and j in the reservoir, (x i , y i , z i ) is the location of neuron i, C i,j is a scalar used to modify the connection probability, and L i,j is a scalar that controls the average length of synaptic connections. The values for both C i,j and L i,j depend on the type (excitatory or inhibitory) of neurons i and j are; the different values for C i,j and L i,j can be found tabulated in table 1.

Energy constraints
Energy constraints were added the reservoir neurons of the LSM in a similar manner to the method of [8].
Neurons were each assigned to an energy pool, and energy was removed from the pool each time one of its member neurons spiked. We set the maximum energy of a pool to be linearly proportional to the number of neurons connected to it with the intention of having the maximum amount of energy in the LSM constant across experiments where the number of neurons assigned per energy pool is varied. For example, a pool with a single neuron connected to it had a maximum energy of one (arbitrary) unit, and a pool with ten neurons had a maximum energy of ten units. Note that in our simulations, pool sizes were not mixed; each pool has the same number of neurons attached to it. Neurons are added to a pool based on distance so that neighboring neurons are more likely to belong to the same pool. To initialize, an energy pool is placed at the same location as a random neuron that does not yet belong to a pool. Afterwards, the closest neurons not connected to any pool are connected to that pool until the desired number of neurons are connected to the pool. This is repeated until all reservoir neurons are connected to an energy pool. See figure 5 for an example of a reservoir with 12 neurons and four neurons per energy pool. Every time a reservoir neuron spikes, it subtracts energy from its connected energy pool. The amount of energy subtracted was varied from simulation to simulation. If the energy pool does not have enough energy, then all neurons connected to it will be prohibited from spiking. Each energy pool regenerates energy at a fixed rate each timestep of the simulation; this was set to 5% of a pool's maximum energy per timestep. In addition, if an energy pool runs out of energy, the accumulated membrane potential of each attached neuron is set to its reset voltage. This is to prevent neurons from firing the instant the pool recovers due to inputs it received while the pool was disabled. Additional reasoning for this decision is that real neurons consume energy when integrating inputs, so they will not be able to integrate inputs if there is no energy to expend. This energy consumption regime leads to hysteresis behavior of a neuron's spiking activity; it will spike at some rate and then be disabled due to energy constraints. After sufficient energy regeneration, there will be a delay before spiking resumes due to the reset membrane potential.

Evaluation methods
In addition to classification accuracy, we studied two metrics to gauge the impact of energy constraints on LSM dynamics: separation and Lyapunov exponent.

Separation
The separation of a reservoir, defined in [20] and expanded upon in [21], is a measure of how far applied inputs spread out in reservoir space. The expanded definition also includes intra-class variance in this metric. The separation quality is defined as [21]: where Sep d is the inter-class distance and Sep v is the intra-class variance of the inputs projected into the reservoir space. The inter-class distance is defined as: where O i is the set of reservoir states for input class i, out of N c input classes. · 2 denotes the L2-norm. C m , the 'center of mass', is defined by [20] as the following: where | · | denotes the cardinality of a set. The intra-class variance of the separation quality is defined as: where ρ i is computed by:

Lyapunov exponent
The Lyapunov exponent is a measure of the 'chaoticness' of a system, or the rate of divergence of the system's state given a small difference in initial conditions. An exponent greater than 0 indicates that the system is chaotic; a small difference in initial conditions will result in a large difference in the final state (divergence). An exponent less than 0 indicates that the system is more ordered; a small difference in initial conditions will result in a small change in the final state and, given enough time, the final states will be identical (convergence). An exponent of 0 indicates that the system is operating on the 'edge of chaos', and is often cited as the optimal region of operation for reservoir computing. The Lyapunov exponent has been estimated by the authors of [22] by introducing a very small initial difference in reservoir state and analyzing its impact on the final reservoir state given identical inputs: where δ o is the magnitude of the initial injected difference and δ ΔT is the difference between the two reservoir states after time ΔT. To compute the Lyapunov exponent estimate in (11), a single initial spike is generated in a random reservoir neuron in the network to form δ o . Then the same input is run through the LSM twice, once with the added spike and once without the added spike, to obtain δ ΔT . This is the method used to estimate all of the Lyapunov exponents shown in this work.

Epileptic seizure detection 4.1.1. Handcrafted LSM (parameter set A)
Accuracy results along with measures of the Lyapunov exponent and the separation of the reservoir when evaluated on the epileptic seizure detection dataset can be seen in figure 6. The LSM was evaluated with various energy parameters; the number of neurons assigned to each energy pool was varied between 1 and 100. The amount of energy consumed per spike was also varied between 0 and 2 energy units, where the reservoir contains a maximum of 100 energy units evenly spread between all energy pools. All simulations were performed with 100 neurons in the LSM's reservoir. The LSM reservoir size was fixed at 100 neurons in an attempt to balance computational cost with the performance. From figure 6, it can be seen that the LSM achieves a maximum in both testing and training accuracy with non-zero energy constraints. This increase in accuracy is correlated with a rise in the separation of the reservoir around the same energy constraint, seen in figure 6(d). For this dataset, a maximal improvement of 5.42% over the case without energy constraints was seen with four neurons per energy pool and an energy consumption of 0.2 per spike. A two-sided Student's T-Test was performed to determine the significance of this improvement, yielding a p-value of 2.92 × 10 −4 , indicating that this result is significant, assuming a significance level of 0.05. It can be seen in figure 6(c) that the Lyapunov exponent of the reservoir is monotonically decreasing, indicating that the reservoir is becoming more and more ordered-that is, there is less and less deviation between final reservoir states given a small initial difference. This intuitively makes sense, as energy constraints tend to drive reservoir activity towards zero, which would cause results to become more and more similar as the energy constraints were increased. In other words, the decreased Lyapunov exponent is likely caused by the attenuation of reservoir activity as energy constraints are increased. It can be seen in the Lyapunov exponent curves that there is missing data in some cases. This is because the Lyapunov exponent was driven to negative infinity, meaning the reservoir responses were the same given an injected spike at the beginning of the evaluation. This is likely due to the energy pooling mechanism and by resetting the membrane voltages of the neurons attached to the energy pool when it is depleted. With more and more neurons per pool and with higher energy consumption, it becomes more and more likely that the membrane voltage accumulation caused by the initial spike will be lost. This is supported by the driving of the Lyapunov exponent to negative infinity sooner as the number of neurons per pool is increased. Figure 7 shows the spiking activity of the reservoir for a part of a sample of the epileptic seizure detection dataset. It can be seen that the energy constraints, depending on the energy pooling regime, serve to attenuate spiking activity, especially very dense spiking activity seen as the bright regions in the figure.

Optimized LSM (parameter set B)
We also performed a random grid search to optimize the LSM hyperparameters. A random grid search was chosen over a standard grid search due to its increased efficiency [23]. The random grid search was done by randomly choosing hyperparameters from a uniform distribution in a specified range (see table 1). 200 random selections of hyperparameters were evaluated with zero energy constraints on the LSM. To reduce computation time, during the random search and in the following experiments with energy constraints, the number of reservoir neurons was set to 60. Of the 200 randomly selected sets of parameters, the reservoir that achieved the highest testing accuracy of 95.5% without energy constraints had hyperparameters as shown in table 1 (set B). Compared to the set of handcrafted parameters seen in table 1 (set A), this reservoir was selected to have a much higher proportion of inhibitory neurons, and a far higher excitatory connection length. As with the handcrafted LSM, an experiment was done to vary the energy consumption of each neuron as well as the number of LIF neurons per energy pool, Results can be seen in figure 8. The maximum for testing accuracy on the seizure detection task occurs at a non-zero energy consumption; an accuracy of 97.5%, which is 2% higher than the unconstrained case. This accuracy was found to occur with four neurons per energy pool  and a consumption of 0.3 energy units per spike. Upon performing hypothesis testing, this raise in accuracy corresponded to a p-value of 0.0156, indicating that the raise in performance is significant. At the same time that the reservoir was able to achieve this accuracy, the number of spikes emitted in the reservoir per sample reduced by an average of 25.9%. Contrasted with the results obtained from the handcrafted LSM, the testing accuracy obtained with this reservoir does not drop off as sharply as energy constraints are increased. This is possibly due to the higher synaptic connection length of neurons in the reservoir.
In addition to varying the number of neurons per pool, we also varied the number of reservoir neurons while maintaining four neurons per pool, which was seen to give the highest testing accuracy based on the results depicted in figure 8. For this experiment, the number of neurons in the reservoir was varied between 100, 200, and 500 neurons while sweeping along the energy consumption per spike. The results from this experiment can be seen in figure 9. For all cases, similar behavior as seen with the 60 neuron reservoir was observed; there were similar trends in the Lyapunov exponent and separation, and the accuracy rose to a maximum at a non-zero neuron energy consumption.

Noisy epileptic seizure detection (parameter set A)
In this work, we also evaluated the impact of energy constraints on the LSM's ability to handle noisy data. White noise with magnitude 50% of the maximum value of each sample was added to the epileptic seizure data. The results are shown in figure 10. It can be seen that the results roughly match those of the noiseless results seen in figure 6. There is a maximum in accuracy at non-zero energy constraints and similar trends in both separation and the Lyapunov exponent, except it can be seen that the Lyapunov exponent begins higher than its noiseless counterpart and the separation is lower. This is expected, as the extra reservoir activity from adding the white noise will make the reservoir states harder to separate as all samples were injected with noise of the same magnitude.
Additionally, it can be noted that both the training and testing accuracy of the noisy version of the epileptic seizure detection task were higher than those of the noiseless version, which is counter-intuitive. It could be that the low accuracy in the noiseless version was an artifact of the relatively low spiking activity, seen in figure 7. The corresponding spiking activity plot for the noisy data can be seen in figure 11, which was collected for the same sample as figure 7, except with the injected white noise. It can be noted that there is significantly more spiking activity in the plot containing the additional white noise. This is possibly due to the increased frequency of bursts of high-magnitude inputs compared to (especially for the healthy-patient EEG data) its noiseless counterpart. This increase potentially allows the LSM to better discriminate seizure activity compared to the relatively inactive noiseless version, as the extra activity would aid in preventing the measure of reservoir activity from reaching 0, which would render classification difficult if this occurred for both classes. If this is the case, however, a similar gain should be obtained through changes in hyperparameters for measuring reservoir activity.
A maximum improvement of 6.17% over the unconstrained case at an energy consumption of 0.25 for four neurons per pool was observed. As with the results on the noiseless EEG detection dataset, a two-sided Student's T-Test was performed, resulting in a p-value of 4.64 × 10 −5 , indicating that this accuracy increase is significant.

Biometric gait identification 4.3.1. Handcrafted LSM (parameter set A)
Results from the biometric gait identification task with the handcrafted LSM can be seen in figure 12. Some similar trends as those with the epileptic seizure detection task can be noted. Maximum testing and training accuracy is achieved at non-zero energy constraints. The maximum testing accuracy increase was found to be 15.82% over the unconstrained case at an energy consumption of 0.2 per spike and 100 neurons per energy pool. As with the epileptic seizure detection task, a two sided Student's T-Test was performed yielding a pvalue of 2.23 × 10 −8 , indicating that this accuracy increase is significant. However, it can be noted that the separation and Lyapunov exponent display opposite trends as seen in the seizure detection dataset-the Lyapunov exponent rises sharply as energy constraints are increased and then tapers off, while the separation of the reservoir decreases, for the most part.
For the results obtained from the biometric gait identification dataset, the increase in accuracy may be attributed to an attenuation of overactivity in the reservoir. Shown in figure 13 is the spiking activity of the reservoir given this dataset. It can be noted that, in the case with no energy constraints, there are over 90 spikes per timestep, indicating that nearly every neuron is firing every timestep, as there are only 100 neurons in the reservoir. As energy constraints are increased, this overactivity is attenuated, potentially allowing the output layer to better discriminate between the reservoir states, though this is not reflected in the separation metric. Figure 13 also shows that increasing energy constraints leads to oscillations similar to those seen in the work of [8]. These oscillations are a result of parts of the reservoir being periodically disabled and re-enabled. When a part of the reservoir spikes sufficiently often, it will be depleted and cease to spike, thus removing inputs to other parts of the reservoir, further reducing the number of spikes occurring in the network. As that part of the network is re-enabled, it yet again begins spiking and contributing inputs to other parts of the reservoir,  increasing the number of spikes in the reservoir. This leads to an oscillatory behavior as this process repeats over time.

Optimized LSM (parameter set C)
In the same manner as the optimized LSM for epileptic seizure detection, 200 random selections of reservoir hyperparameters were evaluated on the biometric gait identification task. The number of reservoir neurons was fixed to 60 in this case as well, in the interest of computational resources. Of these 200 selections, the reservoir with the highest testing accuracy was chosen to undergo further experiments. This reservoir yielded a testing accuracy of 82.4% on the biometric gait identification task in the absence of energy constraints, roughly 10% higher than the unoptimized case which used 100 neurons. The hyperparameters randomly selected for this reservoir can be seen in table 1 (set C). Compared to the handcrafted hyperparameters, the optimized reservoir is more inter-connected; the length of connections between excitatory and both inhibitory and excitatory neurons is greatly increased.   figure 14. A summary of these results can be found tabulated in table 2. In this experiment it was found that the testing accuracy was maximum at non-zero energy constraints; a 2.2% improvement with 60 reservoir neurons per energy pool and 0.15 energy consumption per spike. The p-value for this increase was 0.083, which indicates that the result is not as significant as previous results with this dataset. With energy constraints, the reservoir is still able to achieve similar or slightly higher classification performance with many fewer reservoir spikes. Compared to the results from using the handcrafted parameters, there are similar trends with the Lyapunov exponent and separation; the Lyapunov exponent tends to rise sharply to an extent as energy constraints are applied, while the separation of the reservoir rises some before levelling off.
In addition to varying the energy consumption parameters, we performed a coarse sweep on the number of reservoir neurons. This was done by fixing the number of neurons per pool to the setting that led to the highest testing accuracy during the previous experiment (60). The number of neurons in the reservoir were varied between several values while sweeping across the neuron energy consumption per spike. These results can be seen depicted in figure 15. For this experiment, the regularization parameter used when training the reservoir-to-output weights was not varied from the previous experiment. Given that a considerable amount of overfitting is present in these results with the higher number of reservoir neurons, it is likely that this parameter will also need to be varied in future experiments.

Noisy biometric gait identification (parameter set A)
As with the epileptic seizure detection dataset, white noise of magnitude 50% of the maximum of each sample was injected into the biometric gait identification dataset and this noisy dataset was then used to evaluate the LSM. Accuracy results can be seen in figure 16, while the spiking activities are shown in figure 17. Similar trends were seen as with the noiseless gait identification dataset; the energy constraints tended to attenuate overactivity and introduced an oscillatory behavior, shown in figure 17. A statistically significant increase in testing accuracy was also seen as energy constraints were increased, with a maximum testing accuracy increase of 11.58% observed over the case with no energy constraints with a two-sided Student's T-Test p-value of 1.87 × 10 −8 . This was seen with 100 neurons per energy pool and an energy consumption of 0.65 per spike.

Results for digital LSM
So far, we have made comparisons between the dynamics and classification accuracy of LSMs with and without energy constraints applied. We have also compared our work with another published LSM study [14] for the epileptic seizure detection task. The authors of [14] created an LSM with 60 LIF neurons in the reservoir. They implemented a version of synaptic efficacy introduced in the work of Markram et al [24]. As the output layer of their network, they used an MLP with four hidden neurons and a single output neuron. As input data, they used the epileptic seizure detection dataset and split it into four channels by using bandpass filters with frequencies 0-3 Hz, 4-8 Hz, 8-13 Hz and 13-30 Hz [14]. Polepalli et al also quantized their network-using three bits for intra-reservoir weights, 21 bits for each sample of the input, and 12 bits for the output current of each neuron. For use in training the MLP to classify whether a patient is having a seizure or not, the authors used the number of reservoir spikes that occurred in the last 1 s of each time series of the epileptic seizure detection dataset [14]. As far as we know, this is the only other work that evaluates an LSM on the same binary epileptic seizure detection task.

Handcrafted digital LSM (modified parameter set A)
We modified our handcrafted LSM design to be as similar as possible to the LSM created by Polepalli et al in order to facilitate comparison of results. We reduced the number of hidden neurons in our network from 100 to 60 and performed the same bandpass filtering on the epileptic seizure data. We performed quantization of the input data to 21 bits (by quantizing each channel to Q1.19 fixed point format, with a sign bit) and  the same quantization of the recurrent weights in the reservoir to three bits (we use Q3.0 fixed point and no sign bit). We did not implement an MLP at the output of our LSM and instead kept a single output neuron and trained a set of fully connected weights from the reservoir to the output neuron using linear least-squares optimization. Also, we did not implement any form of synaptic efficacy, nor did we use a sliding spike count as a measure of reservoir activity. We instead used the linearly decaying trace of reservoir activity discussed in section 3. In order to be as comparable as possible to the work of Polepalli et al, we also quantized our measures of reservoir activity, LIF neuron membrane potentials, and pool energy to 12 bit numbers; Q3.9, Q0.12, and Q1.11 respectively.
In tuning the network's performance, we additionally set the average length of synaptic connections to 0.14 times the original handcrafted value, as this was seen to give good results on sweeping this parameter. Also, the timestep size was fixed to 0.01 s rather than using the sampling period of the dataset, as this was seen to give good results. Our implementation of the digital LSM achieves an average testing accuracy of 85.92% without energy constraints, averaged over 20 independent training and testing cycles, each starting with different random number generator seeds. This is comparable to the testing accuracy reported by the authors of [14] (85.1%).
Results from applying energy constraints can be seen in figure 18. We found that applying energy constraints, as seen earlier without quantization and with the single-feature epileptic seizure recognition dataset, leads to an improvement in accuracy. The performance improvement correlates well with a spike in reservoir separation at the same location as the increase in accuracy. We found that the highest observed increase in testing accuracy over the unconstrained case was 4.25%-boosting the testing accuracy to 90.17% on average. This occurred with one neuron assigned to each energy pool and with each spike consuming 0.1 energy units. This increase in accuracy corresponds to a p-value of 0.003 21, indicating significance assuming a significance level of 0.05.

Optimized digital LSM (parameter set D)
An optimized set of the digital LSM reservoir parameters was obtained through the random grid search. The highest testing accuracy observed from all of the sets of random selections was 93.75% in the absence of energy constraints. The hyperparameters that give this result can be seen tabulated in table 1 (set D). As energy constraints were applied, the highest testing accuracy observed was 96.00%, an improvement of 2.25% over the original. This increase in accuracy, with hypothesis testing, was seen to correspond to a p-value of 0.003 26, indicating that the increase in accuracy is significant. This point of highest accuracy increase occurred at an energy consumption of 0.35 with four neurons per energy pool. With these energy settings, the number of spikes in the reservoir was also reduced by 18.3%. A plot of testing and training accuracy along with the Lyapunov exponent and separation for this set of hyperparameters can be seen in figure 19.
Comparing the parameters obtained by random grid search seen in table 1 (set D) with the handcrafted ones (set A), it would appear that the length of all synaptic connections was increased, especially connection length between excitatory neurons. The probability of connection between neurons is also increased for all connection types. In addition, synaptic weights are increased across the board through the synaptic scaling values and α recurrent . The number of inhibitory neurons is also substantially reduced. Overall, this leads to the effect of having a much more interconnected reservoir, especially with excitatory neurons which are allowed to have much longer connections than when manually chosen. This more interconnected reservoir should also spike more often due to the higher synaptic strengths. Figure 19 shows that the same general trend can be seen in both the accuracy and separation; accuracy rises above what it was in the unconstrained case, with a corresponding rise in separation at the same location. These results differ, however, in that the accuracy generally does not drop off sharply as energy constraints are increased. This does happen for the case with one neuron per energy pool because with an energy consumption greater than one, the energy pool will not have a maximum energy large enough to cover the energy cost of a single spike, rendering spiking impossible. The absence of a sudden drop-off in accuracy is not observed  for the other pool settings possibly due to higher inter-connectivity of the reservoir. With a much more highly connected reservoir and higher effective synaptic weights, excitatory spiking activity that would previously have been localized to one location in the reservoir and more susceptible to be shut down due to energy consumption is now spread throughout the reservoir, potentially stabilizing the function of the reservoir at higher energy consumption rates.
Additionally, the value of H was varied using the same quantized reservoir with hyperparameters chosen by random grid search in order to observe the impact of energy constraints at differing reservoir sizes. Results from performing this sweep while evaluating the LSM on the four-channel epileptic seizure detection task can be seen in figure 20. In the creation of figure 20, the number of neurons per energy pool was fixed to 4, as that number of neurons per pool was seen to lead to the best results with a reservoir size of 60 neurons. The number of neurons within the reservoir was varied between 100, 200 and 500 neurons. Larger reservoir sizes were not explored due to their increasingly large computational costs.
In figure 20, the same trend as seen in the case with 60 neurons in the reservoir ( figure 19), can also be seen with larger reservoir sizes. That is, an increase in both accuracy and separation can be observed as the energy constraints are varied. It can additionally be seen that the accuracy at larger reservoir sizes is also larger, which could be attributed to higher separation in the reservoir, possibly caused by the increased dimensionality of the state space. The same two-sided Student's T-Test was performed to test the hypothesis that the accuracy under differing energy constraints was different from the accuracy obtained with no energy constraints.
For 100 neurons in the reservoir, the highest testing accuracy observed was 96.33% at an energy consumption of 0.6 per spike. Without energy constraints, the same reservoir obtained 94.83%. This increase in accuracy corresponded to a p-value of 0.037, indicating statistical significance assuming a significance level of 0.05. With a 200 neuron reservoir, the testing accuracy was improved from 95.17% with no energy constraints to 97.25% at a consumption of 0.35 per spike. Hypothesis testing resulted in a p-value of 0.0015 for this difference. For a 500 neuron reservoir, the highest average testing accuracy observed was 97.75%, improved from 95.08% at a consumption of 0.8 per spike. This case resulted in a p-value of 2.18 × 10 −5 .

Discussion
A high-level summary of results from this work are shown in tables 2 and 3. From table 2, we see that non-zero energy constraints led to a statistically significant increase in testing accuracy for both benchmark tasks with baseline reservoir sizes. This trend also holds across increased reservoir sizes (table 3) and holds for reservoirs that are optimized through a random grid search (although the improvement from energy constraints in the case of the optimized designs is typically smaller). For the epileptic seizure detection task, four neurons per energy pool most frequently led to the best test accuracy. In contrast, results for the biometric gait identification task show that the test accuracy is maximized when the maximum number of neurons per energy pool are used. In addition, and with the exception of the noisy biometric gait identification task, the optimal energy consumption per spike was around 0.25 units (ranging from 0.15 to 0.35). For the noisy biometric gait identification task, the optimal value was much larger at 0.65 units. We also observe an increase in the optimal energy consumption value when moving from the noiseless to the noisy versions of both benchmarks. One explanation for this is that a larger energy consumption per spike will reduce the LSM's sensitivity (and overfitting) to noise since it limits the reservoir activity. Overall, however, the addition of noise had a fundamentally different effect on the biometric gait identification dataset (noise reduced accuracy) than the seizure detection dataset (noise increased accuracy). This can likely be attributed to differences in the datasets themselves. In figure 2, there are differences that are clear between the two classes. Seizure samples contain high-amplitude, high-frequency signals while the healthy samples do not. Intuitively, person-to-person differences in human gait are likely to be more subtle; everyone walks with roughly the same leg motions, but with small differences. It is possible that the presence of the high-frequency information seen in the seizure detection dataset is more difficult to drown out through the addition of noise due to its higher amplitude compared to the more minor variations that might be present in the biometric gait identification dataset.
To gain more insight into the improvement of test accuracy with energy constraints, we plotted the Pearson correlation coefficient between test accuracy and the two reservoir metrics (Lyapunov exponent and separation). The results are shown in figures 21-23 for the regular LSM on the epileptic seizure detection dataset, the regular LSM on the biometric gait identification dataset, and the digital LSM on the epileptic seizure detection dataset, respectively. In the case of the epileptic seizure detection task (figures 21 and 23), we observed that the separation metric has a strong correlation with test accuracy (>0.8) for the handcrafted case across all settings for the number of neurons per energy pool. This correlation decreases when switching to the optimized version of the LSM, but in both cases it is stronger than the correlation with the Lyapunov exponent. In the case of the biometric gait identification task, the separation metric is less strongly correlated with the test accuracy, and the correlation strength tends to decrease more quickly as the number of neurons per reservoir grows. However, as in the epileptic seizure detection task, the separation metric for the biometric gait identification data was usually a better indicator of testing accuracy than the Lyapunov exponent. This general trend that accuracy does not correlate as strongly with the Lyapunov exponent is consistent with findings in the literature that demonstrate the optimal operating region in the ordered-chaos phase space is heavily dependent on the application [25]. That is, for some applications, a more chaotic reservoir will lead to better results, while other applications might require a more ordered reservoir. This is demonstrated in table 2, where the Lyapunov exponent for the epileptic seizure detection task was smaller on average than that of the biometric gait identification task.
The fact that the testing accuracy correlates well with the separation metric is not surprising. This is especially true for the LSM used in this work, with a simple linear readout layer, where linear separability of reservoir Table 3. Accuracy results obtained when sweeping across energy consumption while varying the number of reservoir neurons with a constant number of neurons per energy pool. This was done for both the digital and regular optimized LSM obtained by random grid search used to solve the epileptic seizure detection dataset, as well as the optimized reservoir used to solve the biometric gait identification dataset. Four neurons per pool were used with the epileptic seizure detection dataset, and 60 neurons per pool were used for the biometric gait identification dataset. Asterisks indicate the significance level of the improvement in testing accuracy when energy constraints are added. * : p < 0.05, * * : p < 0.005, * * * : p < 0.000 05. state is critical. We note that in most cases, the energy constraint-driven increase in separation metric does not seem to simply be preventing the LSM from overfitting to training data. If this were the case, we would observe a decreased generalization gap (difference between training and testing accuracy) as energy constraints are applied. Instead, the generalization gap stays approximately constant across different energy consumption values. This is because the training accuracy also increases and reaches an optimum at approximately the same energy consumption value as the test accuracy across most experiments. Therefore, we can conclude that much of the improvement we see in testing accuracies actually comes from an improvement in training accuracy, not an improvement in generalization. However, it is still unclear how the reservoir energy constraints drive an increase in the separability, which in turn drives an increase in training accuracy. One theory, which we plan to explore in future work, is that the energy constraints cause the Sep d and Sep v to decay at different rates, leading to the increase in separation followed by a decrease. This could be modeled by assuming Sep d and Sep v follow a logistic decay model:  where Sep d0 and Sep v0 are the inter-class distance and intra-class variance when no energy constraints are applied, σ(·) is the logistic sigmoid function, γ d1 and γ d2 control the rate and starting location of the interclass distance decay, and γ v1 and γ v2 control the rate and starting location of the intra-class variance decay. Note that according to the model, both Sep d and Sep v will decay to 0 as the energy consumption E c becomes large. This matches our expectation, since very large energy constraints will cause the LSM to stop spiking completely, meaning every state will be identical. Plugging (12) and (13) into (6) gives us one possible way to model the effect of energy constraints on the separation. We show an example of the model in figure 24 with parameters chosen empirically to match the qualitative behavior of the separation vs energy consumption observed in most of our experiments. The key is that the decay rate of Sep v (given by γ v1 ) is larger than that of Sep d (given by γ d1 ), so even though both the inter-class distance and the intra-class variance are decreasing monotonically with energy consumption, there is a global optimum in the overall separation metric at nonzero energy consumption. Based on our results and analyses, we believe that this work could be readily adopted and beneficial in the design of hardware LSMs for edge AI. First, the tunability of the LSM dynamics through a relatively simple hyperparameter (energy per spike) gives the ability to dynamically modify the LSM reservoir behavior. This may be useful in cases where the same basic LSM design is used across different application domains or if the distribution of the data changes over time. This type of simple tuning may also be necessary to compensate for changes in the designed reservoir dynamics due to hardware process variations or wearout mechanisms. We envisage that the hardware overhead associated with imposing energy constraints on the reservoir will be relatively low. For digital designs, a simple up-down counter could be used to keep track of a pool's energy, and for analog designs, the energy pool could be represented as charge on a capacitor. Furthermore, we see our proposed approach as a natural fit for energy-constrained edge AI applications since the reduction in our abstract notion of energy would directly translate to a reduction in real energy. To see this, consider the spike reduction row of tables 2 and 3. In almost all cases, the reservoir parameters that lead to the best test accuracy also cause a large reduction in the total number of spikes that the reservoir produces. This will translate to less energy consumed by the hardware implementation, since each spike will generally require energy to produce and propagate to downstream neurons.

Conclusions and future work
The application of a model of metabolic energy constraints to an LSM was shown to significantly improve testing accuracy when applied to two biosignal processing tasks. For epileptic seizure detection, accuracy increases of 5.42% (without noise) and 6.17% (with noise) were observed with the addition of energy constraints. For biometric gait identification, energy constraints increased accuracy by up to 15.82% (without noise) and 11.58% (with noise). For the seizure detection task, the increased accuracy correlated strongly with the separation of the reservoir, lending some explanation to the phenomenon. However, for the biometric gait identification task, correlations with reservoir dynamics measured with separation and Lyapunov exponent were much weaker. We note that similar accuracy improvements may be obtainable by fine-tuning other hyperparameters in the LSM. However, our goal here was not to improve LSM performance, but rather to study the effects of energy constraints on its dynamics. Furthermore, varying energy constraints may be a simpler solution to adjust the reservoir dynamics compared to tuning other hyperparameters. For example, for an LSM implemented in hardware, modifying parameters like reservoir connectivity may be difficult (e.g. requiring reconfiguration for an field programmable gate array (FPGA)) or impossible (e.g. for an analog/mixed-signal ASIC with fixed topology). Regardless, we believe this paper provides an important step towards designing AI systems where hardware characteristics like energy and performance characteristics like accuracy are not necessarily competing. Avenues for future work are numerous. First, this work should be extended to evaluate the effects of energy constraints on more complex datasets with multiple classes. However, we anticipate that qualitatively similar improvements in performance will be observed (i.e. at non-zero energy constraints) in this case since adding additional outputs to the LSM will not affect the reservoir dynamics. It would also be interesting to implement this type of energy constrained network in hardware in order to analyze the effects on power consumption, battery life, etc. In addition, a method of shaping intra-reservoir weights, which were static during this work, could be developed such that the reservoir could adapt and maintain good performance given imposed energy constraints. Finally, another path for future work is to incorporate a more bio-realistic model of energy constraints, and examine its effects on other neuronal and synaptic behaviors, such as spike-time dependent plasticity.

Data availability statement
The data that support the findings of this study are available upon reasonable request from the authors.