Elsevier

Current Opinion in Neurobiology

Volume 46, October 2017, Pages 120-126
Current Opinion in Neurobiology

Maximum entropy models as a tool for building precise neural controls

https://doi.org/10.1016/j.conb.2017.08.001Get rights and content

Highlights

  • MaxEnt models provide simple interpretable statistical descriptions of neural activity.

  • State-of-the-art MaxEnt models constrain global features of neural activity.

  • MaxEnt models yield novel shuffles, hard or impossible to construct by other means.

  • Adding constraints yields a hierarchy of controls against which data can be compared.

  • Control ensembles can be used for estimating confidence bounds and hypothesis testing.

Neural responses are highly structured, with population activity restricted to a small subset of the astronomical range of possible activity patterns. Characterizing these statistical regularities is important for understanding circuit computation, but challenging in practice. Here we review recent approaches based on the maximum entropy principle used for quantifying collective behavior in neural activity. We highlight recent models that capture population-level statistics of neural data, yielding insights into the organization of the neural code and its biological substrate. Furthermore, the MaxEnt framework provides a general recipe for constructing surrogate ensembles that preserve aspects of the data, but are otherwise maximally unstructured. This idea can be used to generate a hierarchy of controls against which rigorous statistical tests are possible.

Section snippets

From descriptive statistics to probabilistic models

Neural computation arises through the collective behavior of neural populations [1]. What ‘collective behavior’ means in statistical terms is that the distribution of activity configurations (or states) explored by the circuit has nontrivial structure that cannot be explained by statistical properties of individual cells alone. One fundamental challenge for systems neuroscience is to characterize what are the key regularities in neural activity and how they relate to circuit function.

From the

Basic principles for constructing MaxEnt models

The MaxEnt principle provides a compressed description of data that is maximally noncommittal with respect to missing (unspecified) information [24, 25]. It reproduces exactly a set of summary statistics of the data but is otherwise maximally unstructured (see Box 1). To illustrate the basic idea, consider the example of some two-dimensional real-valued data, drawn from a unknown distribution Prob(x, y) (Figure 1a). If we constrain the description of the data to the mean and the standard

Pairwise correlations

One key signature of the collective behavior is the presence of nontrivial pairwise correlations. These are particularly important as they constrain how much information can be encoded in neural activity and the neural readout required to retrieve it [31]. Additionally, since Hebbian forms of synaptic plasticity pick up and positively reinforce correlations in presynaptic and postsynaptic activity [32], changes in correlation structure are a natural measure of circuit-level effects of learning [

MaxEnt as null models

As the most random distribution that satisfies a chosen set of constraints, MaxEnt models naturally provide a null distribution for quantities that are not directly constrained. The idea of using MaxEnt models as null models for hypothesis testing traces back to work by Martignon [28], testing whether effects of a certain order can be explained in terms of lower moments (for instance if the occurrence of triplets of activity can be explained in terms of the firing rates and covariance

Conclusions and future directions

Here we have argued that the MaxEnt framework provides general principles for constructing control ensembles for neural activity. The same principles can also be used for identifying the key regularities captured by different models. In particular, MaxEnt models could serve as a baseline comparison to increasingly popular unsupervised models from machine learning (e.g. restricted Boltzmann machines, hidden Markov models, deep nets). Although these popular techniques have more expressive power,

Conflict of interest statement

Nothing declared.

References and recommended reading

Papers of particular interest, published within the period of review, have been highlighted as:

• of special interest

Acknowledgements

This work was supported by the People Programme (Marie Curie Actions) of the European Union's Seventh Framework Programme (FP7/2007–2013) under REA grant agreement no. 291734.

References (59)

  • J.W. Pillow

    Spatio-temporal correlations and visual signalling in a complete neuronal population

    Nature

    (2008)
  • E. Schneidman et al.

    Synergy, redundancy, and independence in population codes

    J Neurosci

    (2003)
  • E. Schneidman

    Synergy from silence in a combinatorial neural code

    J Neurosci

    (2011)
  • G. Tkačik

    Thermodynamics and signatures of criticality in a network of neurons

    Proc Natl Acad Sci U S A

    (2015)
  • P.T. Sadtler

    Neural constraints on learning

    Nature

    (2014)
  • P. Berkes et al.

    Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment

    Science

    (2011)
  • J. Trousdale et al.

    Impact of network structure and cellular response on spike time correlations

    PLoS Comput Biol

    (2012)
  • M. Helias et al.

    The correlation structure of local neuronal networks intrinsically results from recurrent dynamics

    PLoS Comput Biol

    (2014)
  • G. Tkacik et al.

    Ising Models for Networks of Real Neurons

    (2006)
  • J.S. Prentice

    Error-robust modes of the retinal population code

    PLoS Comput Biol

    (2016)
  • E. Ganmor et al.

    A thesaurus for a neural population code

    eLife

    (2015)
  • B. Doiron et al.

    The mechanics of state-dependent neural correlations

    Nat Neurosci

    (2016)
  • E. Schneidman

    Towards the design principles of neural population codes

    Curr Opin Neurobiol

    (2016)
  • C.M. Lewis et al.

    Recording of brain activity across spatial scales

    Curr Opin Neurobiol

    (2015)
  • P. Gao et al.

    On simplicity and complexity in the brave new world of large-scale neuroscience

    Curr Opin Neurobiol

    (2015)
  • J.P. Cunningham et al.

    Dimensionality reduction for large-scale neural recordings

    Nat Neurosci

    (2014)
  • E.T. Jaynes

    Information theory and statistical mechanics

    Phys Rev

    (1957)
  • S.J. Phillips et al.

    Maximum entropy modeling of species geographic distributions

    Ecol Model

    (2006)
  • G. Tkačik et al.

    The simplest maximum entropy model for collective behavior in a neural network

    J Stat Phys

    (2013)
  • Cited by (27)

    • Building population models for large-scale neural recordings: Opportunities and pitfalls

      2021, Current Opinion in Neurobiology
      Citation Excerpt :

      The main purpose of these models is to ask how much information recorded activity contains about stimuli or behavior, which can be addressed using information-theoretical or decoding approaches. Maximum Entropy models (MaxEnt) provide a principled way to construct probability distributions for spiking activity by finding the least structured distribution that fulfills specific constraints obtained from the data [77]. Typical constraints for MaxEnt models are the firing rates and pairwise correlations between the neurons [79,82,87] and the population synchrony [87].

    • 25 years of criticality in neuroscience — established results, open controversies, novel concepts

      2019, Current Opinion in Neurobiology
      Citation Excerpt :

      Although these three concepts share some features like avalanche size distributions [19] and critical slowing down [75], a definite justification for their interchangable use in the literature remains desirable. Apart from dynamical criticality, neural networks might exhibit statistical criticality, identified by diverging specific heat in maximum entropy models [55,86,87,36,74•,58]. How statistical and dynamical criticality precisely relate to each other in neural networks is yet another topic of open research.

    • Towards the neural population doctrine

      2019, Current Opinion in Neurobiology
      Citation Excerpt :

      These datasets are then passed through a given population analysis, to quantify a null distribution about the extent to which some population structure is expected under the null hypothesis; existing results in both mixed selectivity (see Section ‘Decoding accuracy is more than the sum of its parts’ of this review) and population dynamics (see Section ‘Analysis of neural activity over time reveals computational strategies’) are tested under this framework. See [70–72] for other studies testing and extending these concepts. As recording techniques and computing capabilities continue to improve, experimental and computational studies continue to demonstrate that neuronal populations may in fact be the relevant unit of computation in many brain regions.

    • Neural data science: accelerating the experiment-analysis-theory cycle in large-scale neuroscience

      2018, Current Opinion in Neurobiology
      Citation Excerpt :

      First, there is the concern that novel analyses of large-scale neural data may not be discovering new phenomena, but are rather rediscovering simpler, previously known features of the data that appear new given the novel class of data and algorithms used to investigate these points. Recent work has created statistical hypothesis testing frameworks to enable researchers to quantitatively address this question [84•,120•,121,122], by developing methods to generate datasets that contain these simpler, previously known features but are otherwise random, thus creating a null distribution against which novel large-scale data claims can be tested. Applications to test the presence of linear dynamics in motor cortex [116] and the presence of de-mixed readouts in prefrontal cortex [111] have clarified these previous results [120•].

    View all citing articles on Scopus
    View full text