Efficient information coding and degeneracy in the nervous system

Efficient information coding (EIC) is a universal biological framework rooted in the fundamental principle that system responses should match their natural stimulus statistics for maximizing environmental information. Quantitatively assessed through information theory, such adaptation to the environment occurs at all biological levels and timescales. The context dependence of environmental stimuli and the need for stable adaptations make EIC a daunting task. We argue that biological complexity is the principal architect that subserves deft execution of stable EIC. Complexity in a system is characterized by several functionally segregated subsystems that show a high degree of functional integration when they interact with each other. Complex biological systems manifest heterogeneities and degeneracy, wherein structurally different subsystems could interact to yield the same functional outcome. We argue that complex systems offer several choices that effectively implement EIC and homeostasis for each of the different contexts encountered by the system.


Introduction
The incredible similarity between pearl white and cotton white and the consequent inability to choose one can be discomforting. Nonetheless, an experienced painter who has been regularly exposed to the palette of colors can distinguish them unmistakably well. Such adaptation in responses dependent on the statistical prevalence of specific stimuli can be explained by a fundamental principle called efficient information coding (EIC) (see Box 1 for definitions of important terms). Across a range of neural scales and systems, EIC is accomplished by adaptively matching the response properties of the system to the natural statistics of the stimuli [1e5]. Informationtheoretic analyses [6] provide a strong substrate for formalizing and assessing EIC from the perspective of maximizing stimulus information in system responses.
There are several reasons why EIC is a daunting task. First, the response properties of the system must continually match context-dependent and time-varying stimulus prevalence. Second, multiple timescales associated with various stimulus attributes underscore a need to distinguish temporary environmental fluctuations from persistent changes. Third, adaptations should maintain system stability by recruiting concomitant homeostatic processes that do not hamper EIC. Finally, it is critical to recognize that the rules governing the emergence of EIC could be distinct across scales. Despite these, there is a growing body of evidence that the nervous system robustly accomplishes EIC across all scales.
In this review, we present a unified synthesis with illustrative examples from several species and multiple scales of the nervous system to first demonstrate the ubiquity of EIC. We also build a systematic case that the complexity of the brain is pivotal in its ability to meet the formidable challenges faced in achieving multiscale EIC. Complexity in a system is characterized by several functionally segregated subsystems that manifest a high degree of functional integration when they interact with each other [17]. A characteristic feature of such complex systems is their ability to show degeneracy, whereby structurally different subsystems could interact to yield the same functional outcome [17]. Here, we postulate that degeneracy offers a substrate for simultaneously achieving EIC and homeostasis (Box 1). Our postulate follows from the several degrees of freedom available to a complex system, in terms of the disparate interactions among different subsystems that yield the same functional goal of stable EIC. subcellular response to external stimuli involving its agonists, population activity of neurons constitutes a systems-level code of sensory stimuli. Physiology across scales could be characterized by a well-defined pair of natural stimulus and response, thereby extending the concept of natural stimulus statistics to all biological scales. Such extensions have facilitated the evaluation of EIC as a match between natural stimulus statistics and system responses across all scales, while also accounting for naturally observed dynamics of stimulus attributes [6e9,16,25e27] ( Figure 1).
Exploration of EIC at the systems scale traces its origin to the path-breaking frameworks proposed by Attneave [1] and Barlow [2]. The elegant observation that the response of a neuron in the blowfly visual system matched the cumulative distribution of natural stimuli (luminance contrasts) [3e5] constituted an important step for the EIC framework. Ever since, EIC achieved through the match between neuronal response properties and natural stimulus statistics has been demonstrated across visual [7,9,11,15,28e32], auditory ( Figure 1a) [12,13,33], olfactory [10,34,35], and electrosensory [36,37] modalities. Importantly, although the EIC framework was proposed from a sensory neuroscience perspective, several studies provide lines of evidence for its manifestation in brain regions implicated in cognitive functions such as spatial navigation [14,38,39], value estimation and decision making [40e42]. More generally, EIC could explain states of other parts of the nervous system involved in learning complex task paradigms, perception, and motor command execution [43e47].
Information from the external world is typically represented by action-potential firing properties of individual neurons, through changes in firing frequencies and/or the timing of action potentials. The parameters intrinsic to individual neurons (morphology, ion-channel, and synaptic distributions) critically govern their ability to generate specific patterns or rates of action potentials. Alterations to single neuron properties result in massive changes to information transfer across individual neurons, even if the afferent information impinging on their synapses remain unchanged [14,38,48,49]. For instance, changes limited to ion-channel distributions critically alter the efficiency of spatial information transfer through the rate [38] or phase [14] codes in place cells. Therefore, studies analyzing the efficiency of information transfer in neural responses to external stimuli must account for the physiology of individual neurons as a critical cog in the transformation of natural stimulus statistics to a useable dynamic range of responses [14,38,50,51].
At the neuronal scale, EIC implies a match between single-neuron response properties and the statistics of different attributes of the impinging network activity [50,52e56]. In achieving EIC, single neurons adaptively tune their intrinsic properties (including ion-channel conductances) to match their response properties to Box 1: Definitions of important terms. Efficient information coding: An overarching principle that states that systems can efficiently process their inputs by matching their response properties to their natural stimulus statistics, together maximizing information transfer [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16]. Although certain definitions of efficient coding also encompass energy efficiency, our focus here is exclusively on information transfer efficiency.
Complex system: A system in which smaller parts are functionally segregated or differentiated across a diversity of functions but shows increasing degrees of functional integration when more and more of its parts interact [17].
Degeneracy: The ability of elements that are structurally different to perform the same function or yield the same output [17]. Degeneracy has been shown to be prevalent across all scales of neural systems [18][19][20][21][22][23][24]. Homeostasis: Homeostasis is a self-regulating process by which biological systems tend to maintain stability while adjusting to conditions that are optimal for survival.
Structural redundancy: Structural redundancy in systems refers to structurally identical elements performing the same functions to ensure fail safe operation wherein an identical substitute executes the function in an identical manner when the original component is unable to. Structural redundancy is fundamentally different from degeneracy, where structurally distinct elements are involved in executing the same function. While redundancy can exist even in simple systems, degeneracy is defined to be emergent in complex systems as a consequence of functional integration of functionally segregated structures/subsystems. Structural redundancy does not offer several advantages of degeneracy, including evolvability and robustness to component-specific perturbations [17,20,22].
Information redundancy: Information redundancy in encoding refers to information being redundantly represented by different neurons or networks of neurons [1,2,5]. Information redundancy hampers efficiency, thus making redundancy reduction as an important aspect of achieving EIC.
the natural statistics of dendritic inputs [50e52, 55,56]. Neurons in the hippocampus receive theta-modulated inputs, which translate to strong theta-frequency oscillations in their extracellular and intracellular potentials [57] (Figure 1b). The matched response properties, involving theta-frequency band-pass filtering in the impedance profile [53] and in the spike-triggered average [54,58] along the somatoedendritic axis of hippocampal pyramidal neurons, constitute an example of neuronal-scale EIC (Figure 1b). Efficient information coding across different scales of analysis. (a) Systems scale efficient coding. Left, audio waveforms depicting human vocalization of the phrases 'efficient coding', 'degeneracy', and 'natural statistics' as representative examples of natural stimuli processed by the auditory system. Center, each of the different curves represent the response properties of different neurons in the auditory system. Plotted is the minimal intensity of auditory pure tones at different frequencies required to elicit a neuronal response. The threshold is minimum at the respective characteristic frequency for each neuron, with increases observed on either side. Different neurons respond maximally to different characteristic frequencies, together spanning the range of natural auditory stimuli. Right, dynamical filters that were derived from natural sound statistics matched with the response properties of auditory neurons [12,13]. (b) Cellular scale efficient coding. Left, illustration of extracellular (top) and intracellular (bottom) waveforms, depicting naturally occurring inputs to rodent hippocampal neurons as the animal traversed a linear arena [14,48], manifesting pronounced theta-frequency oscillations. Center, the response properties of neurons in the hippocampal region resemble a band-pass filter, with peak response in the theta-frequency range [53]. Right, dynamical filters derived as the spike-triggered average manifest theta-band characteristic frequency [58]. Inset shows the magnitude spectrum of the spike-triggered average . (c) Molecular scale efficient coding. Left, a synaptic structure showing vesicular release and postsynaptic receptors. The histogram depicts the distribution of the neurotransmitter concentration in the cleft, with the cyan rectangle covering a majority of the naturally observed concentration ranges. Center, the occupancy characteristics of the postsynaptic receptors are aligned with the natural statistics of transmitter concentrations (cyan rectangle), thus allowing for the efficient transfer of information. Right, receptors manifest desensitization, which can be interpreted as a slowly decaying negative feedback loop. In the case of multiple neurotransmitter releases, both the frequency of the releases and the neurotransmitter concentration for each release play important roles in determining the efficacy of dynamical information transfer (inset). The impact of desensitization on the responses is larger when either the frequency or the concentration is high. The natural frequency of neurotransmitter release should be aligned with the neurotransmitter concentration, the receptor occupancy statistics, and the desensitization kinetics for efficient information transfer [100,101].
Extensive studies involving the EIC framework at the molecular scale [6,25e27,59e68] are driven by the recognition that information about an endogenous ligand could be efficiently transmitted by matching the receptor's response properties to natural statistics of the ligand (Figure 1c). The stimulus is defined by the abundance and the dynamics of the ligand, and the response is either the output of the receptor or of a downstream signaling cascade involving receptor activation. A recurring theme across EIC studies at the molecular scale invokes signaling motifs [69], specifically negative feedback loops, that maximize information transfer and alleviate the problem of molecular noise [25,59,60,64,66,70]. Degeneracy in EIC could manifest as system-to-system variability or as an individual system employing distinct context-dependent routes at different instances. A simple illustration of efficient information transfer occurring with these different manifestations is visualized with human communication involving written and verbal forms of different languages, dynamical gestures, and different contexts [17]. The availability of several routes to encode, adapt, match, and respond to persistent changes in natural stimulus statistics offers unique advantages to the system in maintaining robust EIC. Specifically, consider a scenario where a certain component or route fails to perform, owing to the dynamical state of the system or the component's engagement in a different function. Degeneracy then provides a substrate for EIC through recruitment of different components/routes to execute the same task.

Degeneracy supports EIC
Degeneracy as a substrate for simultaneously achieving both EIC and homeostasis is particularly appealing because biological systems continually adapt to noisy dynamical stimuli exhibiting context-dependent natural statistics. Specifically, as the external stimuli is continually changing, there is a need to simultaneously maintain several variables within physiologically plausible levels. The availability of disparate routes ensures that the system has several degrees of freedom to simultaneously achieve these outcomes without cross-interferences. Degeneracy also favors evolvability of efficient coding by virtue of disparate structural components adapting differently to environmental changes, together offering a substrate for adaptive innovations in achieving EIC in a perpetually changing environment [17,22].
Degeneracy forms a reliable substrate to achieve similar information transfer efficiency through several nonunique routes. At the behavioral scale, animals are required to dynamically rely on and effectively use information from various sensory modalities to achieve functional goals such as mate attraction [73] and finding prey [74]. An outstanding example for degeneracy in systems-scale EIC involves information transfer about the identity, abundance, and dynamics of odorants by the olfactory system ( Figure 2b). Through degeneracy, a parametric space spanning activity dynamics of disparate neuronal populations, random synaptic connectivity, and differential olfactory receptor abundance ( Figure 2b) contributes to stereotypic functional outcomes in the olfactory system [34,35,75e78]. With reference to network-scale degeneracy in EIC, response decorrelation (reducing information redundancy is a fundamental principle governing the EIC framework; see Box 1) could be achieved through disparate forms of neural-circuit heterogeneities either individually or synergistically [71,72].
The transformation of synaptic inputs to single-neuron responses is a critical step in the cascade of transformations required for EIC of sensory stimuli by neural responses. Therefore, sensory information transfer is governed by cellularescale parameters that mediate the inputeoutput characteristics of individual neurons. However, efficient coding studies exploring degeneracy with reference to the impact of cellularescale parameters on neural responses to external stimuli have been far and few. There is evidence for degeneracy in the expression of efficient spatial information transfer through rate or phase codes in hippocampal neurons ( Figure 2c). These studies demonstrate that disparate combinations of cellularescale parameters (morphology, synaptic distributions, and ion-channel expression) result in a similar efficiency in spatial information transfer [14,38].
In the molecular-scale parametric space involving receptor identity, downstream signaling motifs, posttranslational modifications on receptor subunits, information typically relates to the abundance and dynamics of agonist molecules [25,26,60,79] (Figure 2d). Although the scope for the expression of degeneracy in EIC is higher at the molecular scale, given the broad parametric space, exploration has been limited. However, there are clear lines of evidence for degeneracy in signaling dynamics involving disparate signaling molecules and pathways [65].

Heterogeneities and EIC
Heterogeneity is an inescapable reality in biological systems. Heterogeneities in the brain span molecular diversity, cell-to-cell, circuit-to-circuit, and animal-toanimal variability in characteristic properties. There are also pronounced functional distinctions in encoding and decoding strategies as well as behavioral and  [75,76]. The abundance of olfactory receptors is also dependent on the odor concentrations, pointing to efficient encoding that accounts for natural stimulus statistics [34,35]. Center, the parametric space includes the abundance of specific olfactory receptors across different animals, neural activity dynamics during odor presentation, the synaptic connectivity across different brain regions. Right, the information space accounts for odor identity (top), odor concentration (middle), and the spatiotemporal dynamics of odor (bottom). There are lines of evidence that disparate connectivity patterns between PN and KC result in stereotypic responses downstream [75,76]. Similar observations about random connectivity and stereotypic function have been made in the mammalian olfactory system as well [77,78]. where receptor activation recruits a feedback motif, eventually resulting in transcriptional changes. Center, the parametric space includes the different subunits that the receptor is composed of, the phosphorylation status of different residues on these subunits, the properties and abundance of downstream signaling molecules, and the signaling motifs recruited by individual receptors. Shown are two different scenarios with disparate subunit composition, phosphorylation status, and signaling motifs (positive vs. negative feedback). Right, the information space involves agonist concentration (top) and dynamics (bottom) and recruits the activation and dynamics of signaling molecules [26, 60,79]. The bottom panel shows two graphs with phasic versus tonic dynamics of agonist encountered by the receptors [25]. There are lines of evidence for degeneracy in signaling dynamics involving disparate signaling molecules and pathways [65].  Plasticity and homeostasis in efficient information coding. (a) Illustrative of a scenario where a change in natural statistics triggers the change in response properties, maintaining efficient information coding. Left: natural visual stimulus is endowed with all orientations. The cortical area is allocated uniformly across all orientations (purple). Right: an artificial intervention, involving rearing of animals in a striped environment, enhances the prevalence of inputs oriented at 135 . The cortical area allocated for 135 is higher (green) than other orientations [89]. Observational approaches to efficient coding assess the relationship between natural stimulus statistics and response characteristics to unveil a match. Interventional approaches, such as the example provided here, add further evidence for efficient coding by demonstrating that targeted manipulations to stimulus statistics introduce matching changes in response properties.  A third perspective explains response heterogeneities to be consequent to EIC, whereby different subpopulations match their response properties to respective natural stimulus statistics. As a broad example, the response profiles of auditory and visual neurons are critically dependent on the natural stimulus statistics they encode [87]. The functional heterogeneity involving two kinds of pyramidal neurons (E vs. I cells) in the electrosensory lobe of electric fishes (Figure 3b) are related to their preferential responses to different afferent stimuli associated with aggressive and courtship communication signals [80]. More subtly, heterogeneity across I-cells has been shown to govern efficient encoding of the quality of the courtship signals [80]. In mice, the dorsal and ventral retinal circuits manifest differential color-opponency, by virtue of distinct spectral characteristics of the cones that form their respective natural stimulus statistics [30].  and a downstream response [59]. Perturbations to the signaling cascades without change in ligand distributions introduce misalignment of the doseeresponse relationship, resulting in response saturation and noise amplification [59,79]. Similarly, perturbations to postsynaptic receptor identity or inputeoutput characteristics, without changes in transmitter statistics, would yield misalignment between the natural stimulus statistics of transmitter abundance and receptor occupancy. Such misalignments of the useful response range result in the saturation of postsynaptic responses and a loss of information about transmitter abundance (Figure 4b).

Adaptation and stability
Together, it is essential to assess perturbations (induced by activity, neuromodulation, or pathology) to response properties that hamper EIC by misalignment of response characteristics with natural stimulus distributions.

Conclusions
Our synthesis spanning several species and multiple scales demonstrates EIC as a generalized biological principle. We argued that biological complexity and ensuing degeneracy are central cogs in the concurrent emergence of EIC and homeostasis. We emphasized the critical roles for parametric heterogeneities as well as dynamics (associated with the stimulus and the response spaces) in improving information transfer. We postulate the interplay between EIC and homeostasis as a universal repeating motif whose balance governs biological systems across scales. The recognition of the ubiquitous nature of these governing principles and explorations focused on degeneracy as a substrate for their concurrent emergence would pave the way for deducing the beneficiary roles of complexity across all biological systems [17]. Tesileanu T, Conte MM, Briguglio JJ, Hermundstad AM, Victor JD, Balasubramanian V: Efficient coding of natural scene statistics predicts discrimination thresholds for grayscale textures. Elife 2020, 9. This study constitutes an extension and generalization of Hermundstad et al., 2014 that suggests that visual sensitivity to natural world is maximal along the directions of highest variability in the feature space (using binary textures). In Tesileanu et al., 2020, they extend this from binary to a larger set of grayscale texture space. With this, they show that the visual sensitivity from human psychophysics data matches the variability in stimulus space, in accordance with the 'variance is salience' hypothesis (Hermundstad et al., 2014), an implication of the efficient coding framework. 14 . Seenivasan P, Narayanan R: Efficient phase coding in hippocampal place cells. Physical Review Research 2020, 2, 033393. This study defined efficiency in spatial encoding, generalizing the EIC framework beyond primary sensory coding. Importantly, it showed that cellular scale parameters and their impact on the single-neuron properties are critical in the maximization of efficient information transfer. A misalignment in the gain of the neuron and the overall synaptic drive results in loss of the useful dynamic range. Quantitatively, the study demonstrates that counterbalancing compensations involving the overall synaptic drive (defined by the excitation-inhibition, E-I balance) and intrinsic excitability (IE) of the neuron drives efficient coding. Thus, the study proposes the use of E-I-IE balance instead of the traditionally employed E-I balance that doesn't account for single-neuron physiology. Assessing the role of different ion-channels in efficient coding, the study demonstrated degeneracy in the emergence of efficient phase coding along with homeostasis of characteristic singleneuron physiology (including firing rate). An important contribution of this study is the assessment of efficient information transfer through a temporal code of external information, rather than the use of rate-based information transfer.

15
. Mlynarski WF, Hermundstad AM: Efficient and adaptive sensory codes. Nat Neurosci 2021, 24:998-1009. The authors in this study present a theoretical framework that assesses the dynamic trade-off between accurate information transmission of stimuli and the ability to detect momentary fluctuations in the environment. Further, they show that the rules derived by accounting for such trade-offs match experimental observations. Importantly, this study provides lines of evidence for the significance of dynamics in efficient coding frameworks. Quantifying information accumulation encoded in the dynamics of biochemical signaling. Nat Commun 2021, 12:1272. A molecular-scale example of the use of EIC principles in analyzing cellular responses to environmental changes, which are encoded in the dynamical patterns of signaling proteins. The study develops a quantitative information-theoretic framework to assess signaling dynamics in the presence of internal noise. Employing different stimulus configurations, the study demonstrates that information-theoretic analysis needs to account for specific temporal phases of stimulus and response dynamics. These quantitative tools could be extended to analyses of EIC in other scales of analyses to account for stimulus/ response dynamics and system-to-system variability. Commun 2020, 11:3481. This study reports that neural circuits within the mouse ventral retina are tuned to extract color information owing to the dominant presence of color-opponent cells in the ventral retina. The study traces the origins for this to the differential localization of cones: the dorsal retina is endowed with two types of cones but most cones in the ventral retina display the same spectral preference. Thus, the complexity of chromatic processing in downstream retinal circuits increased because of the specific types of inputs arriving from the cones, whereby non-linear center-surround interactions created specific color-opponent output channels to the brain. In other words, neurons in the ventral retina tune themselves to extract maximal color-based information resulting in the ventral (but not the dorsal) retinal circuity capable of color opponency aiding in efficient color perception. The efficiently matched circuit-scale processing depended on the natural stimuli they receive, and together enabled robust detection of predators by extracting color information from the upper visual field.

38
. Roy A, Narayanan R: Spatial information transfer in hippocampal place cells depends on trial-to-trial variability, symmetry of place-field firing, and biophysical heterogeneities. Neural Netw 2021, 142:636-660. The relationship between tuning curves and information transfer has been studied across different sensory modalities. This study extends such analyses, involving stimulus-specific information metrics, to spatial information transfer and spatial tuning curves in place cells of the hippocampus. Demonstrating a critical role of single-neuron parameters (ion-channel distribution, synaptic distribution) in spatial information transfer, the study also unveils degeneracy in the ability of the cell to maximize information transfer. The study shows that high trial-to-trial variability in neural responses reduces spatial information transfer.

42
. Polania R, Woodford M, Ruff CC: Efficient coding of subjective value. Nat Neurosci 2019, 22:134-142. A systems-scale example for efficient information coding beyond primary sensory processing. In evaluating EIC, the study considered subjective preference-based decisions as the response and the structure of values in the environment as the stimulus. The authors argue that the decision process involves maximization of information in value representations, while accounting for resource constraints. This study provides an example of EIC by showing that preference-based decisions could be explained by information-maximizing transmission of subjective values in a limited-capacity system.

43
. Koay SA, Charles AS, Thiberge SY, Brody CD, Tank DW: Sequential and efficient neural-population coding of complex task information. Neuron 2022, 110:328-349. e311. An example of systems scale EIC that extends this fundamental principle beyond primary sensory processing. This study recorded activity patterns of a population of neocortical neurons as animals performed a complex dynamic task. The authors report an instance of EIC by demonstrating that task variables with high correlation were represented by modes of neural population that showed low correlations.

47
. Tesileanu T, Olveczky B, Balasubramanian V: Rules and mechanisms for efficient two-stage learning in neural circuits. Elife 2017, 6. This study views the mechanistic basis of vocalization learning in bird songs from an efficient coding perspective. Particularly, the authors show that the activity in the LMAN circuit are tuned to the plasticity mechanisms in the output RA circuit. Further, they show that a mismatch between the LMAN activity and plasticity at the RA synapses impairs learning. The study is thus an example of how the efficient coding framework underlies the shaping of not just the response distributions, but also plasticity rules in non-sensory regions. . Mishra P, Narayanan R: Disparate forms of heterogeneities and interactions among them drive channel decorrelation in the dentate gyrus: degeneracy and dominance. Hippocampus 2019, 29:378-403. Decorrelation of neural responses is a central tenet within the broad EIC framework. This study demonstrates that response decorrelation could be achieved through several routes, specifically involving intrinsic, synaptic, structural, and afferent heterogeneities in neural circuits. This study also offers a unique convergence of cellular-(whereby neuronal characteristic physiology is achieved through disparate structural components) and network-scale degeneracy (showing the emergence of decorrelation through disparate combinations of neural-circuit heterogeneities).

72
. Mishra P, Narayanan R: Ion-channel regulation of response decorrelation in a heterogeneous multi-scale model of the dentate gyrus. Curr Res Neurobiol 2021, 2, 100007. This multi-scale study employs a network that incorporates several forms of neural-circuit heterogeneities in a physiologically relevant manner. The authors demonstrate that perturbation to several different ion channels altered network decorrelation in a differential manner, warning against one-to-one mappings between ion channels and network scale physiology.