Maximum entropy models as a tool for building precise neural controls
Section snippets
From descriptive statistics to probabilistic models
Neural computation arises through the collective behavior of neural populations [1]. What ‘collective behavior’ means in statistical terms is that the distribution of activity configurations (or states) explored by the circuit has nontrivial structure that cannot be explained by statistical properties of individual cells alone. One fundamental challenge for systems neuroscience is to characterize what are the key regularities in neural activity and how they relate to circuit function.
From the
Basic principles for constructing MaxEnt models
The MaxEnt principle provides a compressed description of data that is maximally noncommittal with respect to missing (unspecified) information [24, 25]. It reproduces exactly a set of summary statistics of the data but is otherwise maximally unstructured (see Box 1). To illustrate the basic idea, consider the example of some two-dimensional real-valued data, drawn from a unknown distribution Prob(x, y) (Figure 1a). If we constrain the description of the data to the mean and the standard
Pairwise correlations
One key signature of the collective behavior is the presence of nontrivial pairwise correlations. These are particularly important as they constrain how much information can be encoded in neural activity and the neural readout required to retrieve it [31]. Additionally, since Hebbian forms of synaptic plasticity pick up and positively reinforce correlations in presynaptic and postsynaptic activity [32], changes in correlation structure are a natural measure of circuit-level effects of learning [
MaxEnt as null models
As the most random distribution that satisfies a chosen set of constraints, MaxEnt models naturally provide a null distribution for quantities that are not directly constrained. The idea of using MaxEnt models as null models for hypothesis testing traces back to work by Martignon [28•], testing whether effects of a certain order can be explained in terms of lower moments (for instance if the occurrence of triplets of activity can be explained in terms of the firing rates and covariance
Conclusions and future directions
Here we have argued that the MaxEnt framework provides general principles for constructing control ensembles for neural activity. The same principles can also be used for identifying the key regularities captured by different models. In particular, MaxEnt models could serve as a baseline comparison to increasingly popular unsupervised models from machine learning (e.g. restricted Boltzmann machines, hidden Markov models, deep nets). Although these popular techniques have more expressive power,
Conflict of interest statement
Nothing declared.
References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as:
• of special interest
Acknowledgements
This work was supported by the People Programme (Marie Curie Actions) of the European Union's Seventh Framework Programme (FP7/2007–2013) under REA grant agreement no. 291734.
References (59)
- et al.
Multi-neuronal activity and functional connectivity in cell assemblies
Curr Opin Neurobiol
(2015) - et al.
Long term recordings with immobile silicon probes in the mouse cortex
PLOS ONE
(2016) - et al.
How advances in neural recording affect data analysis
Nat Neurosci
(2011) Open source tools for large-scale neuroscience
Curr Opin Neurobiol
(2015)- et al.
Graphical models, exponential families, and variational inference
Found Trends Mach Learn
(2008) - et al.
Detecting higher-order interactions among the spiking events in a group of neurons
Biol Cybern
(1995) - et al.
Partitioning neuronal variability
Nat Neurosci
(2014) From the neuron doctrine to neural networks
Nat Rev Neurosci
(2015)- et al.
Spikes: Exploring the Neural Code
(1999) - et al.
Weak pairwise correlations imply strongly correlated network states in a neural population
Nature
(2006)
Spatio-temporal correlations and visual signalling in a complete neuronal population
Nature
Synergy, redundancy, and independence in population codes
J Neurosci
Synergy from silence in a combinatorial neural code
J Neurosci
Thermodynamics and signatures of criticality in a network of neurons
Proc Natl Acad Sci U S A
Neural constraints on learning
Nature
Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment
Science
Impact of network structure and cellular response on spike time correlations
PLoS Comput Biol
The correlation structure of local neuronal networks intrinsically results from recurrent dynamics
PLoS Comput Biol
Ising Models for Networks of Real Neurons
Error-robust modes of the retinal population code
PLoS Comput Biol
A thesaurus for a neural population code
eLife
The mechanics of state-dependent neural correlations
Nat Neurosci
Towards the design principles of neural population codes
Curr Opin Neurobiol
Recording of brain activity across spatial scales
Curr Opin Neurobiol
On simplicity and complexity in the brave new world of large-scale neuroscience
Curr Opin Neurobiol
Dimensionality reduction for large-scale neural recordings
Nat Neurosci
Information theory and statistical mechanics
Phys Rev
Maximum entropy modeling of species geographic distributions
Ecol Model
The simplest maximum entropy model for collective behavior in a neural network
J Stat Phys
Cited by (27)
Does the brain behave like a (complex) network? I. Dynamics
2024, Physics of Life ReviewsBuilding population models for large-scale neural recordings: Opportunities and pitfalls
2021, Current Opinion in NeurobiologyCitation Excerpt :The main purpose of these models is to ask how much information recorded activity contains about stimuli or behavior, which can be addressed using information-theoretical or decoding approaches. Maximum Entropy models (MaxEnt) provide a principled way to construct probability distributions for spiking activity by finding the least structured distribution that fulfills specific constraints obtained from the data [77]. Typical constraints for MaxEnt models are the firing rates and pairwise correlations between the neurons [79,82,87] and the population synchrony [87].
25 years of criticality in neuroscience — established results, open controversies, novel concepts
2019, Current Opinion in NeurobiologyCitation Excerpt :Although these three concepts share some features like avalanche size distributions [19] and critical slowing down [75], a definite justification for their interchangable use in the literature remains desirable. Apart from dynamical criticality, neural networks might exhibit statistical criticality, identified by diverging specific heat in maximum entropy models [55,86,87,36,74•,58]. How statistical and dynamical criticality precisely relate to each other in neural networks is yet another topic of open research.
Towards the neural population doctrine
2019, Current Opinion in NeurobiologyCitation Excerpt :These datasets are then passed through a given population analysis, to quantify a null distribution about the extent to which some population structure is expected under the null hypothesis; existing results in both mixed selectivity (see Section ‘Decoding accuracy is more than the sum of its parts’ of this review) and population dynamics (see Section ‘Analysis of neural activity over time reveals computational strategies’) are tested under this framework. See [70–72] for other studies testing and extending these concepts. As recording techniques and computing capabilities continue to improve, experimental and computational studies continue to demonstrate that neuronal populations may in fact be the relevant unit of computation in many brain regions.
Neural data science: accelerating the experiment-analysis-theory cycle in large-scale neuroscience
2018, Current Opinion in NeurobiologyCitation Excerpt :First, there is the concern that novel analyses of large-scale neural data may not be discovering new phenomena, but are rather rediscovering simpler, previously known features of the data that appear new given the novel class of data and algorithms used to investigate these points. Recent work has created statistical hypothesis testing frameworks to enable researchers to quantitatively address this question [84•,120•,121,122], by developing methods to generate datasets that contain these simpler, previously known features but are otherwise random, thus creating a null distribution against which novel large-scale data claims can be tested. Applications to test the presence of linear dynamics in motor cortex [116] and the presence of de-mixed readouts in prefrontal cortex [111] have clarified these previous results [120•].
SiGMoiD: A super-statistical generative model for binary data
2021, PLoS Computational Biology