Little information loss with red-green color deficient vision in natural environments

Summary Inherited color vision deficiency affects red-green discrimination in about one in twelve men from European populations. Its effects have been studied mainly in primitive foraging but also in detecting blushing and breaking camouflage. Yet there is no obvious relationship between these specific tasks and vision in the real world. The aim here was to quantify the impact of color vision deficiency by estimating computationally the information available to observers about colored surfaces in natural scenes. With representative independent sets of 50 and 100 hyperspectral images, estimated information was found to be only a little less in red-green color vision deficiency than in normal trichromacy. Colorimetric analyses revealed the importance of large lightness variations within scenes, small redness-greenness variations, and uneven frequencies of different colored surfaces. While red-green color vision deficiency poses challenges in some tasks, it has much less effect on gaining information from natural environments.


INTRODUCTION
Not everyone has the same color vision. Data from European populations suggest about 8% of men have difficulty with red-green color discriminations and about 0.4% of women are similarly affected. 1 These proportions reflect the genetic origins of most color vision deficiencies. Changes in the cone opsin gene on the X-chromosome array may result in the absence of the long-wavelength-sensitive (L) or mediumwavelength-sensitive (M) cone photoreceptor photopigments or variations in their spectral positions, which, depending on their expression, give rise to the deficiencies of dichromacy or anomalous trichromacy. [2][3][4][5] In dichromacy, a single M pigment and an absent L pigment leads to protanopia and a single L pigment and an absent M pigment leads to deuteranopia. 6 In anomalous trichromacy, two different M pigments and an absent L pigment leads to protanomaly and two different L pigments and an absent M pigment leads to deuteranomaly. 5,6 Absence of the short-wavelength-sensitive (S) pigment, leading to tritanopia, is much rarer and has a different genetic origin. 4,5 The various types of deficiency may be characterized behaviorally and clinically. 5,7,8 Attempts to compensate for their effects with aids such as color filters have had limited success. [9][10][11] Curiously, given the prevalence of red-green color vision deficiency, its impact on vision in the real world has not been quantified in any comprehensive way. As Carvalho et al. noted, 12 most research has concentrated on a specific task, namely food selection, where, in primitive foraging, normal trichromacy offers an advantage for detecting red, orange, or yellow fruits among foliage, [13][14][15][16][17][18][19] or young leaves against a background of mature leaves, 20,21 or flower parts against leafy backgrounds. 22 Other tasks have been considered, for example, detecting blushing or blanching, where normal trichromacy aids social signaling 23,24 ; object detection, where slowly changing targets are more difficult to detect in dichromacy 25 ; and breaking camouflage, where dichromacy offers a compensatory advantage with some scenes and targets, 26,27 but not with others, 28,29 or has a complex role. 30,31 In yet other tasks, red-green color vision deficiency has been found to have less effect than expected on the discrimination of objects with spectral reflectances drawn from natural scenes, 32 once their frequencies of occurrence are accounted for. 11,33 It also has little effect on the judgment of the constancy of surface colors under different lights (illuminant color constancy) with surface spectral reflectances drawn from the approximately perceptually uniform Munsell Book of Color 34-36 or from natural scenes. 35,37-39 It additionally has little or no effect on memory for colored scenes 40 or on discriminating typical illumination changes. 41 To provide a reference level of performance, estimates of the information available with and without color vision deficiency were obtained from subsets of spectra that were more evenly distributed, that is, physically more uniform, than the spectra in individual scenes, and when pooled, covered much larger color gamuts. Since physical uniformity is not the same as perceptual uniformity, estimates were also obtained from the Munsell Book of Color and the Swedish Natural Color System, 59 both of which are approximately perceptually uniform.
It was found that estimated information losses with natural scenes were much less than the information available, an outcome attributed to the large lightness variation within scenes, small chromaticity variation, limited color gamut, and uneven frequencies of different colored surfaces.  53,57 Each color image is rendered as an sRGB image 58 Figure 2 shows mean estimated information losses DI in protanopia, deuteranopia, protanomaly, and deuteranomaly. The top panel is for means over 50 natural scenes, then over 50 approximately uniform subsets of spectra from those scenes, and then for an approximately uniform union of spectra from all 50 scenes, maximizing the gamut available, as detailed in STAR Methods. The middle panel is the same but for the set of 100 natural scenes. The bottom panel is for the approximately uniform Munsell and NCS palettes. Information estimates assumed 2% cone noise. Other noise levels are dealt with later.

Size of information losses
In the top panel, with the set of 50 scenes, mean information losses were 0.4-0.6 bits across the four types of color vision deficiency, equivalent to 6%-11% of the mean information available in normal trichromacy, about 5.9 bits. Losses in the effective number of distinguishable surfaces were proportionally greater, namely 25%-35%, because this number scales with the exponent of the information (STAR Methods). With the 50 approximately uniform subsets of spectra from the scenes, mean information losses increased from 0.4-0.6 bits to 0.5-0.9 bits, and with the approximately uniform union of spectra from all 50 scenes, to 0.9-1.8 bits.
In the middle panel, with the set of 100 scenes, the pattern was closely similar to that with the set of 50 scenes, but mean losses were less, namely 0.2-0.4 bits, equivalent to 4%-7% of the information available in normal trichromacy, about 5.4 bits. Losses in the effective number of distinguishable surfaces were 15%-25%. With the 100 approximately uniform subsets of spectra from the scenes, mean losses increased from 0.2-0.4 bits to 0.4-0.8 bits, and with the approximately uniform union of spectra from all the 100 scenes, to 0.9-1.7 bits.

Figure 2. Losses of information in red-green color vision deficiency
Symbols show the mean estimated losses DI in mutual information 43 in protanopia, deuteranopia, protanomaly, and deuteranomaly. The top panel is for means over 50 natural scenes, 53 then over 50 approximately uniform subsets of spectra from those scenes, and then for an approximately uniform union of spectra from all 50 scenes, maximizing the gamut available, as detailed in STAR Methods. The middle panel is the same but for the set of 100 natural scenes. 54 The bottom panel is for the approximately uniform Munsell 60 and NCS 59  In the bottom panel, with the Munsell and NCS palettes, the ranges of mean losses overlapped the ranges with the approximately uniform unions of spectra. The differences between the Munsell and NCS palettes seem likely to reflect their different chromatic structures. [61][62][63] The presence of vegetation in scenes had a small effect. When the set of 50 scenes was divided into those that were either mainly vegetated or mainly non-vegetated, 64 mean losses were about 0.1 bits larger with mainly vegetated scenes than with mainly non-vegetated scenes.
Corresponding estimates of mean information losses with smaller and larger levels of cone noise are available in Figures S1 and S2. The pattern of losses was largely preserved, that is, the smallest mean losses occurring with individual scenes and the greatest mean losses with approximately uniform unions of scenes and with color palettes. Common to all scenes, spectra, and palettes, information losses were least in deuteranomaly, greatest in protanopia, and intermediate in deuteranopia and protanomaly.

Color variation
Although measures of information and variation capture different properties of distributions, 65,66 it is useful to quantify the variation in the conventional color characteristics of scenes 67 to aid interpretation. Figure 3 shows mean color variance in scenes partitioned across the colorimetric correlates of lightness J 0 , rednessgreenness a 0 M , and yellowness-blueness b 0 M in the approximately uniform color space CAM02-UCS. 68 The organization of the panels is the same as in Figure 2.
Consistent with previous analyses of colored surfaces in natural scenes, 49,69 lightness variation dominated. Averaged over the set of 50 scenes, it accounted for 77% (CI 73%-80%) of the total variance, with the remaining 23% (CI 20%-27%) of chromatic variance consisting of 15% due to yellowness-blueness and 8% due to redness-greenness (percentages have been rounded). With approximately uniform subsets of scene spectra, chromatic variance increased to 27% and with an approximately uniform union of spectra from all 50 scenes to 36%.
Results were similar with the set of 100 natural scenes but with a larger mean lightness variance of 84% (CI 82%-87%) and a smaller chromatic variance of 16% (CI 13%-18%), which presumably contributed to the smaller information losses in Figure 2. The presence of sky in many of the images seems not to be a factor, for when the upper halves of the scenes were removed, the maximum mean information losses were less than 0.03 bits. With approximately uniform subsets of scene spectra, chromatic variance increased to 18% and with an approximately uniform union of spectra from all 100 scenes to 31%.  68 The organization of the panels is the same as in Figure 2. Horizontal bars mark 95% confidence intervals. Do these trends match those for losses of information in Figure 2? There is ordinal consistency in that with the larger chromatic variance in subsets of spectra and still larger with color palettes, the sizes of the mean information losses are correspondingly larger. On the other hand, the chromatic variance in scenes is small (and redness-greenness variance even smaller), which raises the general question of its relevance to information losses.

Test of chromatic contribution
One way to assess the contribution of chromatic variation is to remove its effects. To this end, information losses in dichromacy were compared with those in monochromacy, where with just one cone class, no chromatic information was available. The losses with a single cone class were 3.0-3.9 times more than with two cone classes, confirming the importance of chromatic information in individual scenes. This result, though, is not inconsistent with the relatively small chromatic variance in Figure 3, since the mean losses of 0.2-0.6 bits shown in Figure 2 remained much smaller than the mean available information of 5.4-5.9 bits in normal trichromacy, mentioned earlier.

Prediction by redness-greenness variance
A more specific parametric test of the contribution of chromatic variation is whether information losses increase as the redness-greenness variance in individual scenes increases. The outcome is not foregone since there is no consistent relationship between information and variance 65,66 and both are affected by differences in frequency distributions, 43,70 in this case of the colored surfaces in scenes. Figure 5 shows for the set of 50 scenes the estimated information losses DI with 2% cone noise plotted against the proportions of total variance in each scene due to redness-greenness a 0 M , linearized with the logistic (logit) transformation. 71 The straight lines are linear regressions. The gradients are all positive but as with the mean losses in Figure 2, they decline progressively, that is, for protanopia 0.22 (CI 0.11-0.31), deuteranopia 0.15 (CI 0.07-0.22), protanomaly 0.14 (CI 0.08-0.20), and deuteranomaly 0.08 (CI 0.04-0.12). None of the confidence intervals contained zero. Similarly with the set of 100 scenes ( Figure S3).
Notwithstanding the reliability of the trends, the proportions of variance R 2 accounted for by the regressions are modest. From protanopia to deuteranomaly, values varied from 38% to 21%, with the set of 50 scenes, and from 43% to 38% with the set of 100 scenes. The regressions on lightness J 0 and on yellowness-blueness b 0 M (not shown here) were smaller still, not more than 13% and 4%, respectively, for both sets of scenes.

Article Prediction by chromatic axis
A different test of the effects of chromatic variation is whether information losses vary predictably with the direction of most chromatic variation in each scene, and in particular, whether the losses are maximum when the direction coincides with the confusion axis in protanopia and in deuteranopia. 72,73 As before, the distinction between information and variation 65,66 should be kept in mind. The maxima and minima of the circular regressions fall close to the expected directions, though for both protanopia and protanomaly, there are reliable differences of 16 (CI 8 -22 ) and 18 (CI 7 -25 ), respectively. From protanopia to deuteranomaly, values of R 2 varied from 46% in protanopia to 19% in deuteranomaly.
It is possible that the estimated directions of maximum information loss were biased by the clustering of the major chromatic axes around 90 , the yellowness-blueness axis b 0 M . A control is provided by results with the set of 100 scenes where the major chromatic axes were clustered more around 70 ( Figure S4). The estimated directions of maximum loss were little affected.

DISCUSSION
A focus on specific visual tasks can lead to conflicting interpretations of the impact of color vision deficiency in the real world, ranging from being disadvantageous to broadly neutral or even advantageous. All tasks, however, are contingent on the information available. The present analysis suggests that this information is only a little less in red-green color vision deficiency than in normal trichromacy, by between 4% and 11%, depending on cone noise and the type of color vision deficiency. There are multiple contributory factors, some of which are explored in the following.

Degree of color vision deficiency
Across the four types of red-green color vision deficiency, estimated information losses with the 50 natural scenes ranged from about 0.4 to 0.6 bits out of about 5.9 bits available in normal trichromacy. Estimated The differences between the two sets of scenes seem not to be due to the presence of sky in the set of 100 scenes but to their smaller chromatic variation.
The types of color vision deficiency had predictable effects. Losses were least in deuteranomaly with the largest gap between normal and hybrid L pigment peaks at 559 nm and 549 nm, and they were greatest in protanopia with a normal M pigment peak at 530 nm. 5 Losses were intermediate and similar in deuteranopia, with a normal L pigment peak at 559 nm, and in protanomaly with normal and hybrid M pigment peaks at 530 and 536 nm. The S pigment, with peak at 426 nm, was assumed to be unaltered. These values were chosen to represent the larger discrete differences in pigment peaks, 5,77 which also depend on the frequency, type, and expression of opsin gene arrangements 78 as well as on variations in the density of the lens pigment and macular pigment at the fovea, and in the optical density of the cone photopigment. 79,80 The finding that protanopic losses were always greater than deuteranopic losses confirms the advantage afforded by the approximately 30-nm shift in peak sensitivity from the normal M pigment to the normal L pigment. 81

Role of color gamut and frequency
Early conjectures about the small range of chromaticities in outdoor scenes were based on data from visual spot matching 82 and photographic colorimetry, 83 later confirmed by more comprehensive hyperspectral scene imaging. 69,84-86 Yet as Figure 2 showed, it is not just the size of the gamuts of these scenes but the frequencies of occurrence of colors within those gamuts that govern the information available. Recall that as the frequencies of spectra were made more uniform within scenes, the estimated information available increased and so did the information losses in color vision deficiency, by 1.4-2.0 times. When gamuts were maximized by taking the union of the spectra in scenes and then made more uniform still, the information available increased further and so did the information losses, by 2.4-4.2 times (Figure 2). With the approximately uniform color palettes of the Munsell and NCS sets, estimated information losses were also larger than with individual scenes, by 2.1-5.1 times. Whether defined physically or perceptually, approximately uniform sets of spectra maximizing the gamut available appear to provide an upper bound on information losses.

Chromatic variation vs. lightness variation
One reason for the smaller chromatic variation within scenes and the larger lightness variation, amounting to 80% on average, may be the way natural light is distributed within the environment. 49,[87][88][89][90] Nearby objects and overhead foliage 91,92 can each interrupt the direct beam to produce cast shadows where the reduction in radiant intensity exceeds the change in spectral composition. Another reason may be the iScience Article nature of local surface geometry. 87 Constant surface orientation usually persists over shorter distances than material identity so that changes in reflected intensity exceed those in spectral composition, given that natural surfaces tend to be neither Lambertian nor specular 87 (a potential counter to this explanation is the observation that natural scenes may contain large-amplitude chrominance variations at high spatial frequencies 85 ).
As indicated earlier, though, this does not mean that information losses in color vision deficiency are small simply because there is little chromatic information to lose. When the chromatic component was effectively removed, leaving only lightness information, estimated losses increased by three to four times. The limitations of lightness information were previously noted in an experiment 33 where dichromatic observers discriminated real three-dimensional objects whose spectral reflectances had the same frequencies of occurrence as in some of the scenes in this analysis.
The variation in redness-greenness in scenes also had its predictable effects on estimated information losses, which increased both as the proportion of the variance in scenes increased and as the direction of the major chromatic axis approached the dichromatic confusion axes. Nevertheless, as a predictor, chromatic variation in either magnitude or direction accounted for less than half of the estimated information losses.

Outliers
Although information losses may be small when averaged over natural scenes, it is of course possible to find individual scenes where losses are much greater. Three instances from the set of 50 scenes are shown by the thumbnail images in Figure 5. Two of these scenes had the greatest proportions of redness-greenness variance and were among the more colorful images. Likewise for the set of 100 scenes ( Figure S3). Such scenes may not only be disadvantageous for color deficient vision but lead to failures of color constancy in normal trichromacy. 57,93 Other information estimates Higher-level informational issues have also been examined, for example, the estimated information preserved from scenes under illuminant changes 94,95 ; the coding efficiency of different color spaces 96 ; the effects of fluctuating environmental illumination 97 ; and the information extracted at different levels of the visual pathway. 98 One or many tasks?
If, as argued here, red-green color deficient vision is little disadvantaged in natural environments, it may be construed as evidence of a larger relaxation of the pressure to maintain trichromacy in modern human societies. 28,99 But this is not to say that scene information does not bear on specific tasks requiring color vision, implicitly, or explicitly. 100 Consider the exemplary task of selecting fruit from foliage in the wild. Making that selection involves multiple activities. 18 Together with initially searching a complex optical environment, 87,101 an individual navigating toward an area of interest under direct and diffuse illumination 89,97,102 needs to continuously accommodate changes in the intensity and spectrum of light from reflecting surfaces 90 as relative orientations change 87 and specularities appear and disappear, all of which affect the information available, and eventual success in the task.

Limitations of the study
There are two general limitations to this study, in addition to those already mentioned. First, the magnitudes of the estimated information losses in color deficient vision of 4%-11% depend on the practical minimum level of 2% noise assumed in cones. 103 For levels of noise below and above 2%, losses were, iScience Article respectively, larger and smaller ( Figures S1 and S2). The advantage of natural scenes was, however, retained.
Second, the information estimates remain theoretical and concern solely the information from colored surfaces in scenes. No account was taken of how observers might use that information to guide fixations or interrogate scenes. Encouragingly, the estimate for dichromats of effectively about one-third fewer distinguishable surfaces (average data from Figure 2, top panel) is similar to a reduction in dichromatic observers' experimental discrimination of object spectral reflectances with natural frequencies of occurrence. 33

Conclusion
Despite difficulties with some discrimination tasks, individuals with red-green color vision deficiency lose little information from natural environments. But there is no single determining factor. Rather, it is the large variation of lightness within scenes, the small variation in chromaticity, especially in redness-greenness, the limited color gamut, and the uneven frequencies of different colored surfaces that all lessen the impact of color vision deficiency.

STAR+METHODS
Detailed methods are provided in the online version of this paper and include the following:    Scene spectral data One set of scene images consisted of 50 hyperspectral radiance images of outdoor scenes drawn from the main land-cover classes. Of the 50 scenes, 30 were classified 55,56 as predominantly vegetated, containing woodland, shrubland, herbaceous vegetation (e.g., grasses, ferns, flowers), and cultivated land (fields), and 20 were classified as predominantly nonvegetated, containing barren land (e.g., rock or stone), urban development (residential and commercial buildings), as well as farm outbuildings and painted or treated surfaces. Details of image acquisition, calibration, and processing as radiance images are described elsewhere. 64,109 Each radiance image had dimensions $1344 3 1024 pixels and a spectral range 400-720 nm sampled at 10-nm intervals, giving 33 values at each pixel. The angular subtense of each scene at the hyperspectral camera was approximately 6.9 3 5.3 . To reduce pixel-pixel correlations and non-imaging noise, 57 images were downsampled by spatial averaging over 2 3 2 pixels. Figure 1 shows sRGB 58 rendered color images of the scenes. Almost all the scenes contained color, quantified by the ratio of chroma to lightness. 61 The median ratio was 0.25 and only three scenes had ratios less than or equal to 0.05. None of the images contained sky.
The other set of images consisted of 100 hyperspectral radiance images of outdoor scenes, classified as urban (residential and commercial), suburban, rural, and plant life. Details of image acquisition are described elsewhere. 54 Each image had dimensions $1392 3 1300 pixels and a spectral range 400-1000 nm sampled initially at $1.25 nm intervals and then downsampled to 400-700 nm at 10 nm intervals, giving 31 values at each pixel. For this analysis, a correction was made for the estimated level of dark noise in the images, after which they were downsampled by spatial averaging over 2 3 2 pixels. All the scenes contained color, and the median ratio of chroma to lightness was 0.25 with no scenes having ratios less than 0.10. About 80 of the 100 images contained sky.
For comparison, sets of physically more uniformly distributed spectra in the (bounded) space of radiance spectra were generated in two ways. First, approximately uniform subsets of spectra were obtained from each of the 50 images in the following way (notation adapted from an earlier account 109 ). Let l ij ðlÞ denote the spectral radiance at wavelength l and spatial position ðu; vÞ i , indexed by i in image j and let fl i g j denote the set of all such spectra from image j, where i depends on j. Assume that the wavelength range is the same for each i, j. The required subset fl iðkÞ g j of fl i g j , indexed by k, was then obtained by thinning, 110 that is, by removing spectra l r from fl i g j within a distance d of other spectra l s as d was progressively increased up to a limit for which stable information estimates could be obtained. 111 The distance d between two spectra was defined by the sup metric, dðl r ; l s Þ = sup l ðjl r ðlÞ À l s ðlÞj Þ. The procedure was repeated for each image j.
Second, a single, approximately uniform subset of spectra maximizing the gamut available was obtained from the thinned sets fl iðkÞ g j by taking their union over all j to form the set fl iðkÞ;j g, and then applying thinning once more. A similar procedure was followed with the set of 100 images.

Approximately uniform color palettes
Two large gamut, approximately perceptually uniform sets of reflectance spectra were taken from the set of matt Munsell chips 60 and the set of Natural Color System (NCS) samples. 59 Both palettes span a wide range of chromaticities and lightnesses, 61,62 and subsets of the Munsell palette have been used in practical tests of color discrimination ability, 8 most notably in the Farnsworth-Munsell 100-Hue Test. 112 Despite the artificial construction of the Munsell set, its dimensionality is similar to that of natural reflectance spectra. [113][114][115] Even for a normal trichromatic observer, however, the approximate uniformity of each of the two color palettes holds only imperfectly and mainly locally. [116][117][118] The designs of the two systems are fundamentally different, 61,62 and systematic perceptual differences have been reported between the two, 63 and between their gamuts and the gamuts of natural reflectance spectra. 119 For this analysis, the spectral reflectances of the Munsell set were obtained from measurements by Parkkinen et al., 104 and the spectral reflectances of the NCS palette were recorded in-house with a Konica Minolta CM-2600d spectrophotometer. 119 Radiant spectra from the surfaces were generated by taking the product ll OPEN ACCESS

Cone spectral sensitivities
Radiance spectra were converted 109 into L, M, and S cone excitations based on the Stockman and Sharpe 2 cone spectral sensitivities, lens, and macular pigment data. 108,105 Spectrally shifted absorption spectra were derived from a quadratic loess fit 121 to the normal L, M, and S absorption spectra on a scale of radiance versus log wavelength, with optimal loess bandwidth determined by cross-validation. 106 The normal L, M, and S pigments were assumed to have maximum sensitivities at approximately 559, 530, and 426 nm, respectively. 5, 105,122 In protanomaly, the normal and hybrid medium-to-long-wavelength cone pigments were assumed to have maximum sensitivities at approximately 530 and 536 nm, respectively, and in deuteranomaly, at 559 and 549 nm. 5 The sizes of the hybrid shifts were intended to illustrate their range, not necessarily their prevalence. Optical density was taken to be constant. 79,123 Cone noise Cone signal processing was assumed to be limited by phototransduction noise, 47,124 which varies linearly with background level over a wide range, 125 though the extent of the relevant background may be difficult to specify. 126 The noise distribution was modeled as a Gaussian process whose standard deviation (SD) at each point was specified relative to the cone excitation locally at that point. Previous simulations 10 have found that this process produced similar informational dependencies as a Gaussian process with SD defined relative to the global mean and a Poisson process with an adjusted global mean. The choice of coefficient of proportionality, the Weber fraction, was guided by Stiles' psychophysical increment threshold measurements, 103 which yielded Weber fractions of 0.018, 0.019, and 0.087 for L, M, and S cones, respectively. For brevity, representative values of the relative SD are reported just for L cones, with values for M and S cones scaled appropriately. 10 Results are presented for a relative SD of 0.02, taken as a practical minimum, but other values of 0.01 and 0.05 were also tested.

Mutual information estimates
The information available from scenes, subsets of spectra, and color palettes was estimated with sample points drawn randomly and uniformly from the source data. 109 The spectral radiances and the ll OPEN ACCESS iScience Article