Quality space computations for consciousness

The quality space hypothesis about conscious experience proposes that conscious sensory states are experienced in relation to other possible sensory states. For instance, the colour red is experienced as being more like orange, and less like green or blue. Recent empirical ﬁ ndings suggest that subjective similarity space can be explained in terms of similarities in neural activation patterns. Here, we consider how localist, workspace, and higher-order theories of consciousness can accommodate claims about the qualitative character of experience and functionally support a quality space. We review existing empirical evidence for each of these positions, and highlight novel experimental tools, such as altering local activation spaces via brain stimulation or behavioural training, that can distinguish these accounts

Theories of consciousness include more specific claims about the neural basis of conscious experience [20].Some recent computational theories have achieved greater sophistication and testability by specifying, at least in outline, the computations that are responsible for conscious experience of individual stimuli (compared with processing such stimuli non-consciously).Here, we consider how such theories can account for the hypothesis that phenomenal character includes experiences of subjective similarity.For tractability, we focus on localist, global workspace, and higher-order theories of consciousness.These are sufficiently diverse to make distinct empirical predictions, and differ in how they accommodate existing evidence.

Highlights
The quality space hypothesis about conscious experience proposes that conscious sensory states are experienced in relation to other possible sensory states.
Recent empirical findings suggest that subjective quality space can be explained in terms of similarities in neural activation patterns; however, computational theories of consciousness have had little to say about this qualitative aspect of experience.
Localist, global workspace, and higherorder theories of consciousness can all accommodate a quality space in which subjects experience relations between actually instantiated experiences.
Different theories make distinct predictions for inactivation and stimulation experiments on both local (sensory) and nonlocal (e.g., prefrontal) neural activity patterns.
If experiments find that the whole structure of activation space affects the qualitative character of experience, this can be accommodated by localist and lean workspace/higher-order theories, but is more challenging for full-content global workspace and higher-order theories that require the whole structure to be re-represented.
The standard assumption in consciousness science is that theories can 'mix and match' different accounts of the conscious-non-conscious contrast with different accounts of phenomenal character.We show that, in fact, different theories have more or less difficulty accommodating the quality space hypothesis.In particular, certain versions of workspace and higher-order theories have a problem with assimilating one version of the quality space hypothesis.Hence, perhaps surprisingly, the nature of phenomenal character furnishes evidence for and against some theories of consciousness.

How quality spaces relate to consciousness
Here we consider two ways in which the structure of quality spaces can be reflected in phenomenal character.A modest claim is that the properties that characterise conscious experience fall into families, such that properties within the same family (e.g., colour experiences) bear relations of phenomenal similarity and difference to one another.An analogy is the way in which the level of mercury in a thermometer represents temperature: different levels represent different temperatures, forming an 'organised' representational system [21].The relational structure of temperature readings is latent in a given temperature reading, but is not represented by a single reading (Figure 2A).Under this view, the subject only experiences relations between aspects of their overall experiential state, that is, between aspects that they actually instantiate.There are both empirical and philosophical reasons for thinking that we experience colour relations, for example, in addition to categorical colour experiences [22][23][24].We call this the 'instantiated relations' version of the quality space hypothesis.
A more arresting claim is that the phenomenal character of a single point in quality space, taken on its own, is affected by the overall structure of the quality space (Figure 2E).For instance, Lau argues that the phenomenal character of experience (i.e., 'what it's like' to have an experience) comprises experiencing how that experience is similar to other experiences ( [25], pp. 197-198).Under this Box 1. Probing psychological and neural similarity Arbitrating between theories of the mechanistic implementation of subjective similarity spaces requires mapping between neural and psychological similarity [8,75].Humans in empirical studies can easily and intuitively judge similarity between different stimuli, either with direct ratings of similarity, or within indirect tasks, such as choosing the odd one out (the least similar) of a triad, via graphically arranging similar stimuli together [78], or, in the triplet task, by choosing reference images that are most similar to a target [79].
The result of these empirical investigations is an N × N matrix, where each entry reflects the similarity between all pairs of the N stimuli under test.These data imply a psychological space, that is, how close one stimulus is to another in an arbitrary space.The idea is that what matters for defining a psychological space is not absolute quantities, but distances between different stimuli within a space.Similar matrices can be constructed from objective features of the stimuli, such as their image content or contrast, or from the outputs of machine learning methods, such as convolutional neural networks, applied to the stimulus set.Perceived similarity need not match objective similarity.For instance, for features obeying the Shepherd scaling law, perceived similarity falls off exponentially with distance in psychological space [3].Distances can be computed within different data structures, such as metric distances in a geometric space or non-metric distances between nodes in a graph.A popular technique for inferring a geometric space from empirical data is multidimensional scaling (MDS), which attempts to find a low-dimensional representation of the distances between different stimuli.
A core hypothesis is that what matters for subjective similarity is the distance between two mental or neural representations [8].Representational similarity analysis (RSA) is a commonly used technique that computes the pairwise similarity between neural patterns evoked by different stimuli, to create a representational dissimilarity matrix (RDM [9]).Neural RDMs can then be correlated with behavioural (or model) RDMs to identify correspondences between psychological and neural spaces.Again, what matters here is not the absolute properties of a neural representation, but the relative distance between one representation and another.
The 'structuralist' methodology is a way of applying these techniques to investigate the neural correlates of consciousness.It is promising because it characterises experiences in a more fine-grained way [75,80].For example, while colours can be linearly decoded from activations in both V1 and V4, only the neural activity in V4 reflects the colour quality space, suggesting that activity in V4 and not V1 forms part of the basis of conscious colour experience [81,82].view, if the range over which experiences could occur across that space were different, even in a distant part of space, the subject's experience would be different.An analogy is the way in which the little blue dot on a smartphone map represents not only where the user is at that moment, but also the user's spatial relations to all the other points shown on the screen: if the map were larger or smaller, the content represented would be different.An upshot is that a creature that could only ever experience a single flash of light would not experience it as having any particular colour (cf.[25], p. 198).We call this the 'whole-structure' view.
In the whole-structure version of the quality space hypothesis, the overall structure of the quality space in which an experience E falls affects the phenomenal character of E. The instantiated relations view rejects this: only relations between experiences that are instantiated form part of phenomenal character.This still gives the subject some relational phenomenology (determined by the range of stimuli in play) and the impression that every aspect of experience has both a relational and a categorical phenomenal character (e.g., colour relations as well as intrinsic colour).
In the following sections, we consider how localist, global workspace, and higher-order theories of consciousness can accommodate these two distinct possibilities for a quality space.We consider how each combination of theory and quality space accommodates existing evidence, and makes distinct empirical predictions.

Instantiated relations version of the quality space hypothesis
From the perspective of localist theories of consciousness [12,[26][27][28], the varying activity patterns that support conscious experience are local to sensory systems (including, under local Box 2. Characterising quality spaces and contrasting structuralism Which types of conscious experience fall into quality spaces?It is generally accepted that sensory experiences do, not only the visual [12], but also auditory [83,84] and olfactory [85] experiences.Quality spaces could be restricted to perceptual or sensory experiences, but data on similarity spaces for more abstract categories [67] suggest that the phenomenon extends more widely, perhaps even characterising all conscious experiences.Psychological similarity spaces can be constructed for a range of categories, including facial identity, emotions, and the semantics of thoughts, although for each it is a further question whether the judged similarities are based only on similarity in phenomenal character.
Different kinds of experience appear to fall into distinct spaces.Where subjects find difficulty making consistent similarity judgements across categories, this suggests that the categories do not lie in a common subjective similarity space.For example, two stimuli may be similar in respect of colour and similar in respect of size, but the relative similarity of the colour of one to the size of the other may not be something that figures in experience.Further experiments, for example, on the consistency of similarity judgements within and between categories or sensory modalities, are needed to confirm this phenomenological intuition.
Experienced similarity requires some shared dimension or dimensions of similarity.For example, colour experiences are arranged into a quality space structured by three underlying dimensions: hue, brightness, and saturation.Experiences can be rated as similar separately along each of these dimensions.Even if experiences that share no common dimension, such as size and colour, are not experienced as bearing a relation of similarity/dissimilarity, humans may nonetheless be able to make judgements of similarity across experience types, for example between sound and shape [86].
The existence of quality spaces must be distinguished from structuralism about consciousness.Structuralism is a metaphysical claim about the nature of conscious experience.It is the claim that what makes a particular conscious experience E a conscious experience of a specific type is a matter of the relations of E to other experiences [87].As a metaphysical claim, structuralism can be distinguished from structuralist methodology for studying consciousness (see Box 1 in the main text).The viability of structuralist methodology is independent of the truth or otherwise of structuralism as a metaphysical thesis [75].For example, while the whole-structure version of the quality space hypothesis sits naturally with structuralism, even the instantiated relations version of the quality space hypothesis, or only the existence of quality spaces, allows us to use the similarity structure of experience to investigate its neural realization, even when those similarities are a contingent property and not part of what constitutes an experiential type.The strong claim of structuralism is not required to allow us to use similarity structure to infer that V4 but not V1 is responsible for realising colour experience (see Box 1 in the main text).

Glossary
Conscious experience: mental state or event with a phenomenal character; thus, there is something it is like for a subject to have a conscious experience.Full-content higher-order or workspace theory (of consciousness): sensory information only becomes conscious if it is appropriately recapitulated in other regions of the brain (in a higher-order representation or first-order re-representation/barcode).Global neuronal workspace theory (GNWT): consciousness is constituted by the global broadcast of sensory representations through their being re-represented in a (prefrontal/parietal) central workspace.Higher-order state space HOSS) model: consciousness is constituted by a higher-order mechanism that assigns a sufficiently high degree of reliability to first-order representations based on their magnitude or precision.First-order contents are gated into consciousness without being re-represented by the higher-order mechanism; a lean theory.Higher-order theory (of consciousness): consciousness is constituted by higher-order representations of mental states.Higher-order thought theory (of consciousness): consciousness is constituted by the content of thoughts about first-order (e.g., sensory) states; a full-content theory.Lean higher-order or workspace theory (of consciousness): additional processing is needed for sensory information to become conscious, by being broadcast or reality tagged, but the sensory content need not be meta-represented or re-represented.Perceptual reality monitoring (PRM): higher-order theory of consciousness in which sensory representations become conscious when indexed by a higher-order mechanism and categorised as reflecting external reality (rather than noise or imagination).PRM mechanisms are held to operate implicitly and continuously, and are not subject to volitional control.Phenomenal character: subjective or experiential properties of a conscious experience; what it is like to undergo the experience (called 'subjective character' by some theorists).recurrence theory, recurrent activity).Localist theories can readily accommodate the instantiated relations version of the quality space hypothesis.
One potential computational instantiation of this view is provided by population coding models of perception [29][30][31].Under these models, different neural populations are tuned to different sensory attributes (e.g., motion direction or colour), and the combination of different tuned activation functions indexes a particular location in a (high-dimensional) perceptual space.The similarity between population codes is proposed to underpin psychological similarity [32].Evidence for this view comes from recent studies mapping the psychological similarity of different visual stimulus dimensions (shape, colour, background, function, and total similarity) onto fMRI and magnetoencephalography (MEG) data.By partialing the contribution of each of several representational dissimilarity matrixes (RDMs), different subregions within the ventral visual cortex were shown to correlate with different aspects of subjective similarity [15] (Figure 1B).Similar analyses can be performed on data from higher-resolution methods, such as single unit recording [33,34] and intracranial electroencephalography (IEEG) [11,13,35,36], and RSA allows data from different resolutions (e.g., fMRI and single-unit recordings) to be compared in a common space [37].
One variant of the population-coding account holds that the brain is doing 'linear algebra' by combining activity along different (perhaps arbitrary) perceptual dimensions, with different levels of firing along different dimensions realising specific experiences.Studies of the neural encoding of faces in macaque inferotemporal (IT) cortex highlighted how individual faces are represented by duration from different categories [13].The pattern of activation across recording sites in occipital and ventrotemporal cortex contains sustained and stable (invariant) information about the visual percept throughout the duration of the stimulus.Notably, while overall activity magnitude and firing pattern changes over time, there is a stable maintenance of neural dissimilarity between two stimulus category representations.(B) Location of representations underlying different facets of perceived visual similarity in the human brain [15].A searchlight approach was used to create fMRI representational dissimilarity matrices (RDMs) in a sphere around each voxel, which were then correlated with perceived similarity RDMs for different features of a set of 118 object images (maps for shape, function, and color are shown; see [15] for other similarity dimensions).(C) Neuronal ensemble activity from the macaque ventrolateral prefrontal cortex (VLPFC) during briefly presented visual stimuli [51].The identity of each stimulus could be robustly decoded from PFC population activity even under rapid serial visual presentation (RSVP) conditions, demonstrating a rich representation of visual content in prefrontal areas.Reproduced from [13] (A), [15] (B), and [51] (C). a linear combination of neural activations within a 50-dimensional 'face space' [38].Notably, using binocular rivalry, both the consciously perceived and suppressed faces could be decoded from inferotemporal neuronal activations, but they occurred in distinct subspaces [39].

Trends in Cognitive Sciences
This view explains why local patterns of activation, which localist theories claim are the basis of specific contents becoming conscious, have a qualitative character.First, they exist in a similarity space, implemented in a neural activation space, the dimensions of which, as with size or colour, give the experience its particular character (just as a particular level of mercury in a thermometer falls in a similarity space with other levels of mercury).Second, relations between actually instantiated experiences (points in space) are themselves experienced.Since activation in response to a given input in practice changes over time, the subject will enjoy a range of different relations between experiences (Figure 2C).Thus, the actually instantiated relations may be rich.
An empirical prediction of the instantiated relations view is that interventions that alter the neural encoding space (e.g., preventing neurons encoding a particular dimension from firing) would not In the whole-structure hypothesis, the whole structure of the quality space affects the phenomenal character experienced at each point in the space, with the effect being constitutive not only causal.This is illustrated by the red point on the map, a currently experienced point in the space, having relational connections with other points in the space.(F) With a constitutive effect, deactivating or changing the activation space is like cutting out part of a paper sheet: the experience changes even though the local pattern of activation, the red point, stays the same.(G) A merely causal effect is like cutting out part of a rubber sheet: the experience changes because the space deforms, changing the focal activity pattern.(H-J) Illustration of how localist and global workspace/higher-order theories can accommodate quality spaces.Each subpanel illustrates points in high-dimensional neural activity space, here shown in three dimensions (e.g., the firing rate of three distinct neural populations).(H) Localist theories account for quality space in terms of location in a neural activation space.(I) According to full-content theories (higher-order or workspace), relational aspects are only experienced to the extent that those relations are recapitulated in the further states in which sensory information is re-represented or meta-represented.(J) According to lean theories (higher-order or playground), the relational content of sensory states forms part of experience in virtue of a global process without having to be recapitulated.
change the phenomenal character of items that remain within the purview of the restricted neural space.One piece of evidence in support of this view is the finding that there are 'null' dimensions of neuronal manifolds along which changes in the stimulus do not affect neural responses [38,40].

Whole-structure version of the quality space hypothesis
There is also some empirical support for the whole-structure version of the quality space hypothesis, as treated by localist theories of consciousness.This view has been defended by Malach ([12], p. 7], who suggests that lateral cortical connectivity provides a neural substrate for the relational geometry.According to this account, local lateral connections in sensory brain areas implement a quality space for each sensory domain.Since representational geometry is defined by the neural activation patterns triggered by inputs, the brain network responsible will also depend on the strength of inputs to an area and on feedback connections.In turn, lateral connectivity itself (and, therefore, the quality space) will be modulated by top-down neuromodulatory connections.
Indirect evidence for such a view is provided by recent data showing that relative population activity patterns in visual cortex and their relational geometries are sustained during extended viewing of visual stimuli [11,13,35] (Figure 1A).Notably, while overall activation magnitude rapidly adapts, conscious perceptual content covaries with the multivariate neuronal pattern, which obeys constant similarity relationships with other stimulus-induced neuronal patterns.A notable feature of these neuronal patterns in posterior visual areas is that the representational geometry remains stable and unchanging over time, unlike the findings from prefrontal areas, which show more transient responses [13].Thus, according to these data, what appears to matter for the experience of specific perceptual content is the geometry in which neural patterns are arranged, rather than absolute features of activation magnitude (Box 1).Intriguingly, similar representational geometry is observed in the early layers of deep neural networks trained on visual tasks [41].However, such data could be (computationally) explained with the kind of population-coding dynamics described in the previous section, given that the same relational geometry can be instantiated with different neuronal implementations [42].
It might appear strange that relations to merely possible experiences should affect phenomenal character, but this view makes that un-mysterious, since the wider brain network, which is actual, not just potential, is a realiser for this (extremely rich) phenomenal character.An analogy is the way in which a glowing blue dot on a map represents where I am now.It represents my relations to all the other points on the map.The variable element is the blue dot, but the background map, which stays fixed, is a realiser for a very rich relational content.
A key prediction of the whole-structure view of qualitative character (unlike the instantiated relations view) is that, if one intervenes to change the structure of the activation space, and then manages to re-instantiate the same local pattern (e.g., using optogenetic reactivation methods), then what is experienced by the subject should differ even though the distributed pattern of activation remains the same.Testing this hypothesis requires more than just a causal change to the pattern of activation produced in response to input, for example following adaptation or learning.The claim here is that the whole space makes a constitutive difference: what is experienced will change even if the space is not deformed (similar to cutting out part of a paper map; Figure 2F).Some intriguing psychophysical evidence for this view was obtained by Song et al. [43].By using a Hebbian training protocol to strengthen lateral connections between two points of visual space, the perceived distance between untrained locations was also altered.These findings suggest that a local change in the strength of lateral connections is sufficient to alter experience of visuospatial relations.

Quality spaces within workspace and higher-order theories of consciousness
We now turn to global workspace and higher-order theories of consciousness.Workspace accounts, such as the global neuronal workspace theory (GNWT), posit that what is required is the broadcast or sharing of sensory information into a central workspace [44][45][46].Higherorder theories propose that what is important for consciousness is the re-representation or monitoring of first-order sensory information [47][48][49].
David Rosenthal's higher-order thought theory holds that phenomenal character is fixed by the content of a higher-order thought that represents a perceptual representation [2].Stanislas Dehaene's version of GNWT calls for sensory information to be re-represented and broadcast to become conscious [44,46].The broadcast representations are supposed to act like a 'barcode' that encodes the broadcast content in full.Unlike higher-order theories, the broadcast representations are first-order.However, both theories are committed to phenomenal character depending on the content being carried in full by the higher-order or broadcast representation: they share a commitment to a full-content view of what is required for a content to be conscious.Since this commitment generates shared issues for accommodating the quality space hypothesis, we discuss these two theories in the same section, despite the important differences between them.Other higher-order and workspace theories require less of the second component, as we shall see.We discuss these lean accounts in the following section.
Full-content higher-order and workspace theories According to Rosenthal's higher-order theory, qualitative character is given by higher-order thoughts that represent mental qualities in terms of locations in a quality space ([2], p. 383).He also holds that the subject can appreciate the relation between two mental qualities that occur concurrently and, thus, discriminate between them, but not between experiences that occur at different times ( [2], p. 383).Instantiated relations contribute to phenomenal character, but the whole structure of quality space does not.
Dehaene's version of GNWT also subscribes to a full-content view of what is required for a content to become conscious.Evidence for a full re-representation of perceptual content in the global workspace comes from recent intracranial recording studies in both humans [13,35] and nonhuman primates [50][51][52].These studies used either passive viewing or binocular rivalry to reveal the coding of multiple visual dimensions in prefrontal cortex (PFC), even when subjective reports were not required [50] (Figure 1C).However, it remains unclear how such decoding of contents relates to the representational spaces or population codes that have been identified in sensory areas (e.g., the face-space population code in IT cortex discussed earlier) or whether it recapitulates the RDM of perceptual similarity judgements.Compared with sustained representations in sensory areas, the PFC shows more transient coding of stimulus content at stimulus onset [13,35], together with the emergence of content-invariant visibility or 'phenomenal magnitude' signals [53][54][55].
Thus, it is unclear whether and how the PFC instantiates a quality space, as opposed to 'reading out' task-dependent aspects of organised, localist representations.One perspective on this question is provided by studies of the task dependence of PFC representations.Unlike in sensory areas, the dimensions of the neural code representing particular sensory features can be radically compressed or elongated depending on whether those features are task relevant [56][57][58].Such a flexible reformatting of sensory representations in PFC suggests that global representations are less tied to the phenomenal character of conscious experience.However, whether phenomenal character is also unchanged in such scenarios remains an empirical question, given there is evidence for attention-and task-dependent changes in phenomenal character as measured by similarity judgments [4,59].
A theory intermediate between the full-content and lean views is Lau's perceptual reality monitoring (PRM) variant of higher-order theory [60].PRM starts from the observation that the mere presence of a sensory signal does not create conscious experience; instead, consciousness ensues when such signals are categorised as reflecting external reality.PRM proposes that this difference is underpinned by a downstream higher-order mechanism that indexes and interprets the first-order representations, akin to a discriminator in a generative adversarial network (GAN [61]).
Extending PRM to encompass quality spaces, Lau ([25], pp.197-211) and Lau et al. [62] propose that a higher-order monitoring mechanism has access to the nature of neuronal coding in sensory cortices, similar to how feature-specific attention targets specific neuronal populations [63,64].
The similarity between higher-order indices encodes implicit knowledge of the structure of sensory spaces.It is the similarity between indices, not the similarity between the first-order states indexed, that gives rise to phenomenal character.PRM readily accounts for experienced similarity between actually instantiated experiences, to the extent that their relations are recapitulated in PFC [25,62], but faces more difficulty accommodating the whole-structure view.
A prediction of both full-content views is that aspects of the similarity structure embodied in local/ first-order neuronal representations that are not structural features of the higher-order or broadcast state will not contribute to phenomenal character, even though they may affect behaviour.This prediction is difficult to test, as it requires interfering simultaneously on putative local (sensory) and global neuronal representations, but may be possible with optogenetic manipulation approaches, or the use of behavioural training approaches in combination with brain imaging.Indirect evidence may be obtained by asking how noninvasive brain stimulation [e.g., transcranial magnetic stimulation (TMS)] or lesions to PFC affect the structure of perceptual similarity judgments.Previous studies demonstrated the recruitment of abstract grid and place codes in PFC and hippocampus when navigating olfactory [65] and auditory [66] perceptual spaces.Such neuronal codes may support implicit access to relations between different points in quality space, as posited by PRM.However, the mapping between grid codes and conscious experience remains to be determined.One concern is that, if a similarity space is explicitly learned, these abstract codes may reflect the requirements of the navigation task, rather than reflecting perceptual experience [67].
Lean higher-order and workspace theories A leaner version of higher-order theory is provided by the higher-order state space (HOSS) account, which proposes that a higher-order state 'tags' first-order perceptual content with a degree of reliability or experiential magnitude [68].Contents become conscious when the level of first-order magnitude or precision crosses a certain threshold.Under this view, the contents of perception are determined by first-order activations within sensory systems (similar to localist accounts), and no aspect needs to be re-represented to form part of conscious experience.Instead, the higher-order state acts to 'gate' particular contents into conscious experience when they are considered reliable enough [69,70].
The contribution of a magnitude signal to reality monitoring has gained recent empirical support through observations that, in near-threshold perception, signals from perception and imagination jointly determine reality monitoring judgments [71].However, a lean higher-order theory in which PFC and parietal cortex only support higher-order (and not also first-order) states would not predict findings that transient activations in these regions also show rich relational representations of perceptual content [13,35,70] and that prediction errors on perceptual magnitude in medial PFC also encode features of perceptual content [72] (Figure 1C).
It is straightforward for the HOSS model to account for the instantiated relations version of the quality space hypothesis.It can also readily accommodate the whole-structure hypothesis, since it does not call for recapitulation of first-order contents, unlike PRM, which requires the whole structure of quality space to be recapitulated in similarities between higher-order indices.
If the experience E of a point in quality space is indeed affected by the whole structure of the space, then an activity pattern-plus-brain-network is a good candidate to realise that experience, with a higher-order gating mechanism being responsible for the whole of that first-order content becoming conscious.
Lean workspace accounts also hold that, while an additional process is needed to make sensory representations globally available, hence conscious, the first-order contents need not be rerepresented.One example is the 'global playground': a broad network of areas among which sensory representations are shared and maintained for several hundreds of milliseconds, thus offering wider cognitive possibilities compared with automatic unconscious processing, but with no specific agenda [73].The global playground does not require contents to be re-represented to become conscious.Nevertheless, representations in the global playground have a different functional signature compared with non-conscious representations.
Similar to localist theories, and like the HOSS model, the global playground can readily account for conscious experience of relations of subjective similarity.When a representation in a similarity space becomes conscious, the experiential property exists in a space of possibilities, in analogy to the range of levels of mercury in a thermometer.The nature of the experience is determined by its location in that quality space, which is realised by the location of a pattern of neural activity in neural activation space.Furthermore, relations between aspects of experience are realised by relations between occurrent neural patterns and, thus, also form part of the phenomenal character of experience, in line with the instantiated relations view.Lean workspace theories can also accommodate the whole-structure view, in the same way as lean higher-order theories, since no re-representation is required.
A prediction of lean higher-order accounts is that, if the relations between stimulus features remain stable (correlated), even in the face of absolute changes in activation, such a configuration will be more likely to pass a gating mechanism and, thus, become conscious.Features that are each too noisy to become conscious might, if the noise is correlated, be gated into consciousness together.Some preliminary evidence for this hypothesis comes from experiments showing that more stable neural activity patterns are associated with conscious perception [74].A prediction of the global playground view is that conscious representations have a similarity structure that is different from, and richer than, that instantiated by non-conscious representations [39,[75][76][77] (Box 1).

Concluding remarks
We have considered how neural similarity structures can be integrated into current computational accounts of consciousness.Localist, higher-order, and workspace theories can all account for the fact that experiences occur in a quality space.They can also readily accommodate the hypothesis that subjects experience relations between actually instantiated aspects of experience.If empirical investigations find, more arrestingly, that the whole structure of quality space affects the phenomenal character of each individual experience, that finding can be accommodated by localist theories, and also by lean higher-order and lean workspace theories.It is more problematic for full-content higher-order and full-content workspace theories, since they would require the whole structure to be re-represented.
Notably these views make distinct predictions for inactivation and stimulation experiments on both local (sensory) and global (e.g., prefrontal) neural activity patterns.These techniques are currently most accessible in animal models.While consciously experienced qualities are difficult to

Outstanding questions
Are properties of stimuli that are represented non-consciously (e.g., chromatic stimulation as opposed to perceived colour) represented in similarity spaces or in some other way?If in similarity spaces, are there systematic differences between conscious and non-conscious similarity spaces?More ambitiously, is there a characteristic type of similarity space that goes along with states being conscious?
Is experience of actually instantiated relations between experiences restricted to those occurring at a moment, or does it extend to relations between aspects of experiences occurring at nearby times?Does noise/stability in relations in sensory content affect whether those contents become conscious (in line with a quality space version of lean higher-order theories of consciousness)?
Can behavioural adaptation approaches be extended to ask about the role of adaptation of local connectivity in other kinds of sensory experience (both visual and non-visual)?How can higher-order index views be tested at the level of neural implementation?What would a neural signature of an index and its associated similarity space look like?Must similarities be recapitulated in higher-order indices to be experienced (as claimed by PRM), or is it sufficient that first-order similarity structure is tagged as real and thereby gated into experience (as claimed by HOSS)?
How do quality spaces depend on the temporal structure of consciousness, and the potential for extended windows of integration?How do quality spaces change over time, given that neural similarity structures are themselves distributed in space and time?How are relations encoded for simultaneously presented features (as anticipated under the instantiated relations view)?probe in animals, a first step would be to combine inactivation methods with behavioural techniques for probing psychological similarity.Our overview also provides an organising framework for interpreting correlational data from studies comparing neural similarity structures obtained for conscious versus non-conscious representations in the human brain (see Outstanding questions).
Computational accounts of conscious experience to date have had little to say about the relational aspects of phenomenal character.We hope that this survey sets an agenda for incorporating the quality space hypothesis into existing theories of consciousness, at a time when emerging new data in the neurosciences make addressing this question both pressing and tractable.

Figure 1 .
Figure 1.Quality spaces and neural similarity.(A) Intracranial recordings from patients viewing stimuli of variable

Figure 2 .
Figure 2. Theoretical options for capturing quality space.(A) The instantiated relations hypothesis about the nature ofsubjective quality space holds that experiences lie at a point along one or more dimensions, similar to the level of mercury in a thermometer.(B) Relations between currently active experiences (instantiated relations) are also experienced, but not the whole structure of the space.(C) Since each aspect of experience will in practice vary slightly over time, the subject will experience a whole family of such instantiated relations.(D) Switching analogy to points on a map, on the instantiated relations hypothesis, only relations between actually instantiated experiences are reflected in consciousness, indicated by the currently active red points on the map.(E) In the whole-structure hypothesis, the whole structure of the quality space affects the phenomenal character experienced at each point in the space, with the effect being constitutive not only causal.This is illustrated by the red point on the map, a currently experienced point in the space, having relational connections with other points in the space.(F) With a constitutive effect, deactivating or changing the activation space is like cutting out part of a paper sheet: the experience changes even though the local pattern of activation, the red point, stays the same.(G) A merely causal effect is like cutting out part of a rubber sheet: the experience changes because the space deforms, changing the focal activity pattern.(H-J) Illustration of how localist and global workspace/higher-order theories can accommodate quality spaces.Each subpanel illustrates points in high-dimensional neural activity space, here shown in three dimensions (e.g., the firing rate of three distinct neural populations).(H) Localist theories account for quality space in terms of location in a neural activation space.(I) According to full-content theories (higher-order or workspace), relational aspects are only experienced to the extent that those relations are recapitulated in the further states in which sensory information is re-represented or meta-represented.(J) According to lean theories (higher-order or playground), the relational content of sensory states forms part of experience in virtue of a global process without having to be recapitulated.