Introduction

Recent innovations such as optogenetics and neuroimaging enable us to characterise the relationship between the activity of neurons and their system-level behaviour. These technical innovations call for theories that describe neuronal interactions and reveal the underlying principles. In physics, electromagnetism, the general theory of relativity, and the quantum field theory have been consolidated within the rigorous and holistic framework of a gauge theory [1]. Here, we propose that a gauge-theoretic formalism in neurosciences might not only provide a quantitative framework for modelling neural activity but also show that neuronal dynamics across scales—from single neurons to population activity—are described by the same principle. This paper suggests that if we could formulate a gauge theory for the brain—or cast an existing theory as a gauge theory—then many aspects of neurobiology can be seen as consequences of fundamental invariance properties. This approach could clarify the intimate relationship between apparently distinct phenomena (e.g., action and perception) and, potentially, offer new tools for computational neuroscience and modelling.

This paper is comprised of three sections. The first provides an informal description of gauge theories in physics. The second focuses on the biological features required for a gauge theory for the brain. In the third section, we discuss the three ingredients of a gauge theory: a system equipped with symmetry, some local forces, and one or more gauge fields. To illustrate the approach, we use the principle of variational free energy minimisation as a candidate for a neuronal gauge theory. This principle says that every aspect of the brain, from neuronal activity to synaptic connection strengths, can be described as a process that minimises variational free energy, where variational free energy measures the probability of sensory inputs, under the brain’s model of how those inputs were caused. The variational aspect of variational free energy comes from its derivation from variational calculus, which is at the heart of gauge theories and information geometry, which are the focus of this article.

At its simplest, variational free energy reduces to prediction error or surprise in exactly the same way that estimating the parameters of a statistical model can be reduced to minimising the sum of squared error. We will see later that when physical laws, such as Hamilton's principle of least action, are applied to variational free energy, the resulting behaviour looks very much like statistical inference, lending the brain—and indeed any self-organising system—the look and feel of a system that is trying to estimate or infer the causes of its (sensory) exchanges with the world. In fact, advanced statistical analyses of data minimise variational free energy, where negative free energy approximates Bayesian model evidence. Effectively, minimising variational free energy corresponds to maximising model evidence. This means that if the brain minimises variational free energy it is inherently Bayesian; hence the Bayesian brain hypothesis [2–5].

We provide a glossary of technical terms (S1 Glossary) and series of appendices (S1 Appendix; S1–S5 Text) that offer a technical account of the free energy formalism, gauge theory, and their relationship. These appendices unpack the key conceptual points for technically-minded readers. Although formulating the free energy principle as a gauge theory may have important implications for new inference schemes that can be used to analyse data (i.e., infer the structure and parameters of models of empirical data), we focus on the implications for neurobiology.

I. Gauge Theories in Physics

A gauge theory is a physical theory that predicts how one or more physical fields interact with matter. Every gauge theory has an associated Lagrangian (i.e., a function that summarises the dynamics of the system in question), which is invariant under a group of local transformations [6]. Consider Newton’s laws of motion in an inertial frame of reference (e.g., a ball in an empty room). These laws are valid at every point in the room. This means that the dynamics and the Lagrangian do not depend on the position of the ball. In this system, the dynamics will be invariant under translations in space. These transformations—that preserve the Lagrangian—are said to be equipped with gauge symmetry. In short, a symmetry is simply an invariance or immunity to changes in the frame of reference [7].

Take a rotating frame of reference (e.g., the Earth) and call it the system. During the rigid rotation of the Earth, the distances between every town remain invariant. As Earth rotates, one can say that we have a gauge transformation that does not change the distance between the cities (S1 Appendix). This means that the system preserves its symmetry (c.f., the laws of Nature), despite the action (or gauge transformations) of global forces (acting on every point of the system). This action corresponds to a global transformation of the system—a transformation that preserves invariances; in this instance, distances. There is, however, another possibility: for example, compare the Earth and moving objects on its surface (e.g., the water in the ocean). When the motion of water is described relative to the Earth, its inertia (that is, its tendency to move in a straight line) distorts its motion. This phenomenon is called the Coriolis effect: rather than flowing directly from areas of high pressure to low pressure—as they would in a nonrotating world—currents tend to flow in a clockwise sense (with respect to the direction of travel) in the Northern hemisphere and counterclockwise in the Southern hemisphere. The Coriolis effect is thus a curvilinear motion in a rotating (noninertial) frame of reference. A gauge transformation that is not constant, i.e., varies with spatial location, is referred to as a local symmetry. One or more local forces acting on just a few parts of the system produce local transformations.

Why is the symmetry preserved, despite the perturbing action of local forces? This is because the local forces are balanced by another force—called the gauge field. Note that when the global symmetry of the system is preserved despite local forces, there is a local deformation, and the symmetry is said to be locally broken, or at best, hidden. In a system equipped with global symmetry, it is thus possible to find a local breaking of symmetry that can be “repaired” by the gauge field. Let us again take the example of the Coriolis forces. For someone who only uses inertial frames of reference, the Coriolis forces do not exist; they are just forces we invent to pretend we are in an inertial system when in fact we are not: they are practical, because they allow us to continue using Newton’s laws. Albert Einstein discovered something similar; the laws of special relativity are not valid when gravitational forces (which are due to energy and momentum) are present. However, by allowing energy and momentum to bend space–time, he could retain special relativity, at least locally, making the gravitational forces disappear. For Einstein, gravitational forces are as unreal as the Coriolis forces; they are forces we have to invent when we choose an inappropriate frame of reference. See S1 Appendix that illustrates these ideas using a simple pendulum example.

In summary, gauge theory is relatively straightforward: one has a Lagrangian that prescribes equations of motion. These equations of motion are generally invariant under changes in a spatial frame of reference (e.g., a straight line is a straight line everywhere on a sheet of paper); however, if we stand back and see that motion (a line in space–time) changes with position (e.g., a straight line on a sheet of paper that has been bent), we can extend the global symmetry to a local symmetry by introducing compensatory fields or forces—the gauge fields. The notion of bending or curvature here is important, because manifolds that shape trajectories in the space of probability distributions are necessarily curved. This is important for systems that minimise variational free energy (S1 Text) and related Bayesian brain theories [8]. In fact, we shall see below that quintessentially cognitive faculties like “attention” can, in principle, be derived using gauge theory.

II. Gauge Theories for the Nervous System

Gauge theories originate from physics; however, they could be applied to countless fields of biology: cell structure, morphogenesis, and so on. Examples that lend themselves to a gauge-theoretic treatment include recent simulations of embryogenesis [9] and the self-organisation of dendritic spines [10]. Although these examples appear unrelated, both can be formulated in terms of a gradient ascent on variational free energy. In other words, we may be looking at the same fundamental behaviour in different contexts [11]. Here, we focus on the central nervous system (CNS). Can we sketch a gauge theory of brain function? The answer rests on identifying—in biological systems like the brain—the tenets of a gauge theory; namely,

1. A system equipped with a continuous symmetry that can be conserved.
2. The presence of local forces acting on the system (a group of local transformations), where a gauge field renders its Lagrangian invariant under such transformations.

When attempting to establish what aspect of CNS function might be understood in terms of a Lagrangian, the variational free energy looks highly plausible [12]. The basic idea is that any self-organising system, at nonequilibrium steady-state with its environment, will appear to minimise its (variational) free energy, thus resisting a natural tendency to disorder. This formulation reduces the physiology of biological systems to their homeostasis (and allostasis); namely, the maintenance of their states and form, in the face of a constantly changing environment.

If the minimisation of variational free energy is a ubiquitous aspect of biological systems, could it be the Lagrangian of a gauge theory? This (free energy) Lagrangian has proved useful in understanding many aspects of functional brain architectures; for example, its hierarchical organisation and the asymmetries between forward and backward connections in cortical hierarchies [13]. In this setting, the system stands for the brain (with neuronal or internal states), while the environment (with external states) is equipped with continuous forces and produces local sensory perturbations that are countered through action and perception (that are functionals of the gauge field).

In summary, the free energy formalism rests on a statistical separation between the agent (the internal states) and the environment (the external states). Agents suppress free energy (or surprise) by changing sensory input, by acting on external states, or by modifying their internal states through perception [12,14]. In what follows, we try to show that the need to minimise variational free energy (and hence achieve homeostasis) acquires a useful logical mathematical formalism, when framed as a gauge theory.

III. Sensory Entropy as a Lagrangian

The variational free energy formalism uses the fact that biological systems must resist the second law of thermodynamics (i.e., a tendency to disorder), so that they do not decay to equilibrium. In a similar vein to Maxwell's demon, an organism reduces its entropy through sampling the environment—to actively minimise the self information or surprise of each successive sensory sample (this surprise is upper bounded by free energy). By doing so, it places a bound on the entropy of attributes of the environment in which it is immersed. Variational free energy operationalises this bound by ensuring internal states of the system become a replica (i.e., a generative model) of its immediate environment. This can be regarded as a formulation of the good regulator hypothesis [34], which states that every good regulator of a system must be a model of that system.

We know that a gauge theory would leave the Lagrangian invariant under continuous symmetry transformations. Therefore, a gauge theory of the brain requires the interaction among three ingredients: a system equipped with symmetry, some local forces applied to the system, and one or more gauge fields to compensate for the local perturbations that are introduced. The first ingredient is a system equipped with symmetry: for the purposes of our argument, the system is the nervous system and the Lagrangian is the entropy of sensory samples (which is upper-bounded by variational free energy, averaged over time). The local forces are mediated by the external states of the world (i.e., through sensory stimuli). The gauge fields can then be identified by considering the fact that variational free energy is a scalar quantity based on probability measures. Let us see how.

From differential geometry treatments of probability measures (S3 Text), it is known that the manifold traced out by sufficient statistics of a probability measure is curved in nature [35,36]; more specifically, it has negative curvature and is therefore endowed with a hyperbolic geometry. Barring technical details, it suffices to understand that a functional (e.g., variational free energy) of such probability measures is not flat but induces a curved (Riemannian) landscape. Moving along such a curved manifold requires a measure of distance that corresponds to the distance between two distributions. Again, differential geometry tells us that such a measure of distance is provided by the Fisher information metric (Cencov’s characterisation theorem) that quantifies the amount of information that a random variable carries about an unknown parameter. Technically, the Fisher information represents the curvature of the relative entropy (Kullback-Leibler divergence). Put simply, distances in the curved geometry of sufficient statistics—that define variational free energy—correspond to the relative entropy in going from one point on the free energy manifold to another.

Instead of using classical results from differential geometry [35–39], we will motivate the gauge formulation of variational free energy by asking an empirical question: how does neuronal activity follow the steepest descent direction to attain its free energy minimum? In other words, how does it find the shortest path to the nearest minimum? As the free energy manifold is curved, there are no orthonormal linear coordinates to describe it. This means the distance between two points on the manifold can only be determined with the help of the Fisher information metric that accounts for the curvature. Algebraic derivations (S3 Text) tell us that, in such free energy landscapes, a Euclidean gradient descent is replaced by a Riemann gradient, which simply weights the Euclidean gradient by its asymptotic variance.

In the free energy framework, when the posterior probability is approximated with a Gaussian distribution (the Laplace approximation; S4 Text), perception and action simply become gradient flows driven by precision-weighted prediction errors. Here, prediction errors are simply the difference between sensory input (local perturbations) and predictions of those inputs based upon the systems internal states (that encode probability distributions or Bayesian beliefs about external states that cause sensory input). Mathematically, precision-weighted prediction errors emerge when one computes the Euclidean gradient of the free energy with respect to the sufficient statistics. In a curvilinear space, the precision-weighted prediction errors are replaced by dispersion- and precision-weighted prediction errors. This says something quite fundamental—perception cannot be any more optimal than the asymptotic dispersion (inverse Fisher information) regardless of the generative model. In statistics, this result is known as the Cramér-Rao bound of an estimator. In other words, the well-known bound (upper limit) on the precision of any unbiased estimate of a model parameter in statistics emerges here as a natural consequence of applying information geometry. In the context of the Bayesian brain, this means there is a necessary limit to the certainty with which we can estimate things. We will see next, that attaining this limit translates into attention. See Box 1 for an overview.

Notice that the definition of a system immersed in its environment can be extended hierarchically, wherein the gauge theory can be applied at a variety of nested levels. At every step, as the Lagrangian is disturbed (e.g., through changes in forward or bottom-up sensory input), the precision-weighted compensatory forces change to keep the Lagrangian invariant via (backward or top-down) messages. In the setting of predictive coding formulations of variational free energy minimisation, the bottom-up or forward messages are assumed to convey prediction error from a lower hierarchical level to a higher level, while the backward messages comprise predictions of sufficient statistics in the level below. These predictions are produced to explain away prediction errors in the lower level. From the perspective of a gauge theory, one can think of the local forces as prediction errors that increase variational free energy, thereby activating the gauge fields to explain away local forces [40]. In this geometrical interpretation, perception and action are educed to form cogent predictions, whereby minimization of prediction errors is an inevitable consequence of the nervous system minimising its Lagrangian. Crucially, the cognitive homologue of precision-weighting is attention, which suggests gauge fields are intimately related to (exogenous) attention. In other words, attention is a force that manifests from the curvature of information geometry, in exactly the same way that gravity is manifest when the space–time continuum is curved by massive bodies. In summary, gauge theoretic arguments suggest that attention (and its neurophysiological underpinnings) constitutes a necessary weighting of prediction errors (or sensory evidence) that arises because the manifolds traced out by the path of least free energy (or least surprise) are inherently curved.

Conclusions

The importance of the reciprocal interactions among different scales and between the nervous system and the world has been repeatedly emphasised here and elsewhere. A quantitative formulation of this holistic aspect of information processing in the brain is clearly needed and has been framed in terms of variational free energy minimisation or, more simply, as a suppression of prediction errors encountered by an agent that is actively sampling its environment [12,14]. In this essay, we have reformulated the resolution of prediction error (or surprise) as a gauge theory for the nervous system. Such geometrical formulations endow variational inference with symmetry breaking, enabling invariance with respect to the generative model; irrespective of the model’s parameterization [41,42].

The free energy formulation enables us (or our brains) to compare a variety of hypotheses about the environment given our sensory samples. So what does a gauge formulation add? After scoring a variety of models according to their variational free energy, one selects the best generative model that explains the data at hand. If symmetry transformations can be evaluated for the best performing model, afforded by the symmetry group, we can possibly go a step further and ask what other models could explain the data with the same model evidence? Such symmetry transformations therefore furnish us with a family of models that explain the data with equal probability [43–45]: see Fig 1 for a schematic illustration of this idea. The challenge of such an approach would be to deduce the symmetry groups of the probability distributions encoded by neuronal activity (S5 Text). If this was possible, one could identify the class of models explaining data using a single inference methodology. From the perspective of biological self-organisation, this means that there are potentially many different models that do the same job of predicting sensory data. It is tempting to associate the equivalence classes afforded by symmetry groups above with the biological species—all doing an equally good job of modelling their (local) environments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Invariance of inference—a proposal.

A univariate normal model space (of the equivalent data space) whose coordinates are sufficient statistics. The orange circles represent two model spaces, where minimization of variational free energy leads to the appropriate model (blue circles in each case). It might be that one parameterisation (blue outer circle for model space 1) is more suitable than another. Notice that using the Riemann gradient instead of the Euclidean gradient automatically guarantees gauge-invariance by breaking the symmetry within a model. But what about between-model symmetries? After the optimal model has been selected, symmetry transformation could enable one to derive a range of models (black circles) that describe the data equally well (in terms of log model evidence or negative free energy). These can be obtained by using the intrinsic geometry of the gauge field (the Levi-Civita connection) and the symmetries afforded by the variational free energy.

https://doi.org/10.1371/journal.pbio.1002400.g001

Supporting Information