Quantiﬁcation and Standardized Description of Color Vision Deﬁciency Caused by Anomalous Trichromats—Part II: Modeling and Color Compensation

A color compensation scheme has been developed to enhance the perception of people with color vision deﬁciency (CVD) and for people su ﬀ ering from anomalous trichromacy. It is operated within the MPEG-21 Multimedia Framework, which provides a standardized description of CVD. The basic idea behind the proposed color compensation consists of simulating the path of human color perception. As such, compensated color is realized by relying on the spectral cone sensitivities of the human eye and the spectral emission functions of the display device. For quantiﬁed color compensation, the spectral sensitivity of anomalous cones has been modeled according to the deﬁciency degree of the standardized CVD description. The latter is based on the error score of a computerized hue test (CHT), developed in Part I of our study. Given the anomalous cone spectra, the reduction of error score on the CHT after color compensation was measured in each deﬁciency degree. The quantitative relationship of color compensation with the error score is linearly regressed, based on the deﬁciency degree with the least error score after color compensation as well as the error score before color compensation. License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


INTRODUCTION
Today, one can easily access multimedia contents with highquality colors, the ability to perceive colors correctly becomes an important user aspect.However, color vision deficiency (CVD) may be a significant barrier when trying to offer transparent access to visual contents on the multimediaenabled devices.
According to the universal multimedia access (UMA) paradigm, it is desirable that everyone becomes capable of easy and equal accessing of all types of multimedia, anytime and anywhere. In this context, providing transparent accessibility to multimedia content for people having a CVD is challenging. To address this problem, the MPEG-21 multimedia framework [1,2] has recently standardized a normative description of CVD characteristics for content adaptation. The CVD description includes a textual and numerical characterization of the severity degree of CVD [2]. However, in order to apply color compensation according to the CVD description standardized by the MPEG-21 multimedia framework [2], the way to generate the standardized CVD description with a generic color vision test remains a challenging issue.
In some of our previous works [2,3], a novel color compensation scheme has been developed, resulting in improved color accessibility for people suffering from CVD. The color compensation scheme has been designed to operate in the context of the MPEG-21 multimedia framework. However, the quantitative study based on the MPEG-21 CVD descriptions has not been delineated, that is, it is necessary to know the amount of color for which compensation is required.
Quantification of the total error scores of conventional color testing, such as the Farnsworth-Munsell 100-Hue (FM100H) test, has been studied in the literature [4][5][6]. However, the previous quantification approaches have been performed on the D-15 panel test [4,5], which is a shorter version of the FM100H test for only allowing to detect dichromacy. Quantification by the FM100H test can be used for diagnosing anomalous trichromacy. However, thus far, no work has been done to quantify the total error score using the FM100H test.
As such, several issues are to be addressed in this paper: (1) modeling the spectral cone sensitivities according to the deficiency degree standardized by the MPEG-21 multimedia framework; (2) measuring the degree of deficiency of anomalous trichromats; (3) developing a method to measure the deficiency degree by using existing color vision tests.
This paper is Part II of the study that quantifies color vision for anomalous trichromats, based on error scores using a computerized hue test (CHT). In Part I of this study, we have seen that CVD degrades linearly according to the degree of deficiency. Due to the linear characteristics of the deficiency degree in the standardized MPEG-21 CVD description, this observation is important for matching the error scores of the CHT to the standardized CVD description. Thus, it can be expected that the CHT could provide a quantitative measure of anomaly in color vision. In Part II of our study, we discuss color compensation for anomalous trichromats and in particular the associated quantification of color compensation according to a standardized description of the severity degree of deficiency.

Cone fundamentals of anomalous trichromacy
Protanomalous trichromats have normal S and M cone pigments, but the peak sensitivity of the L cone pigments, denoted by L cone pigments, is shifted to a shorter wavelength relative to that of the normal L cone pigments [7][8][9]. The deuteranomalous trichromats have normal S and L cone pigments, but the peak sensitivity of the M cone pigments, denoted by M cone pigments, is shifted to a longer wavelength compared to that of the normal M cone pigments [7][8][9]. Figure 1 shows the spectral sensitivities of the LMS cones in the visible wavelength from 400 nm to 700 nm, which were originally measured Smith and Pokorny [10] and DeMarco et al. [11]. The wavelengths of the peak sensitivity were 440, 543, and 566 nm for normal trichromats, 440, 543, and 553 nm for protanomalous trichromats, and 440, 560, and 566 nm for deuteranomalous trichromats. As seen in Figure 1, which is estimated by DeMarco et al. [11], the L cones and M cones are separated by 10 nm for the average protanomalous trichromats, and the M cone and L cone are separated by 6 nm for the average deuteranomalous trichromats.

Anomalous cone modeling according to deficiency degree in MPEG-21
By the CVD description of the MPEG-21, the severity of anomalous trichromacy is expressed in numerical degrees. The numerical degree in this paper comes from MPEG-21  DIA (digital item adaptation) [1]. The main goal of MPEG-21 DIA is to define a multimedia framework to enable transparent and augmented use of multimedia across a wide range of networks and devices used by different communities [1]. Table 1 presents the medical terms of CVD by the description of color vision deficiency with either textual degree or numerical degree or both [1,2]. The severity degree of anomalous trichromacy can be represented by a textual degree of "Mild" and a numerical degree going from 0.1 to 0.9. Anomalous trichromacy can be quantified into 9 numerical degrees (from 0.1 to 0.9 in increments of 0.1 step) in terms of the degree of deficiency. It is noted that the numerical degree of 1.0 denotes dichromacy, which can be also represented by a textual degree of "Severe".
For color compensation, we should be aware of the spectral sensitivity of anomalous cones. But there is no reference to the spectral sensitivity of anomalous cones at different degrees of deficiency. Therefore, the modeling of the spectral sensitivity of anomalous cones is required for color compensation. In order to model the anomalous cone, two important aspects are considered: one aspect is the shift of the peak sensitivity curve, while the other aspect is the shape of the curve. The range of shift amount of peak sensitivity for the anomalous cone is known to range up to 20 nm [8]. We quantify the shift of peak sensitivity linearly by 2 nm steps from 2 nm to 18 nm. It is assumed that there might not be a continuum of peak cone sensitivity. For easy adaptation of color compensation to real applications, we simply applied a linear quantization of the shift amount of the peak cone sensitivities. Table 2 shows the deficiency degree assigned to a particular shift of the peak sensitivity (λ abnormal estimated peak ). Next, we model the cone spectral sensitivity curve for all visible wavelengths. The shift amount at each individual wavelength is modeled based on the shift of peak sensitivity and the shape that is obtained by the Smith and Pokorny anomalous cone models [7,10]. Figure 2 shows the spectral shift at each individual wavelength obtained from the Smith and Pokorny anomalous cone models. Smith and Pokorny   [7,10] used the template for the normal M cone pigment to estimate the shape of the protanomalous L cone pigment, and they used the template for the normal L cone pigment to estimate the shape of the deuteranomalous M cone pigment [11]. The resultant sensitivities have irregularities over the visible wavelength, as seen in the bigger shift in the longer wavelengths. This phenomenon was also supported by M. Neitz and J. Neitz [13].  degree, the spectral shift amount of anomalous cone sensitivity at a wavelength λ, denoted as Δλ estimated (λ), is estimated as where λ abnormal measured peak is peak sensitivity measured by Smith and Pokorny's anomalous cone (see Table 2), λ abnormal estimated peak is peak sensitivity for an anomalous cone to be estimated, Δλ measured (λ) is the spectral shift measured by Smith and Pokorny's anomalous cone at wavelength λ (see Figure 2), and Δλ estimated (λ) is the spectral shift to be estimated at wavelength λ. Figure 3 shows the modeled anomalous cones.   Figure 4: Overall procedure for color compensation.

COLOR COMPENSATION
The overall process of color compensation is seen in Figure 4: (1) the visual characteristics of anomalous trichromacy is specified based on the symptom associated with the type and severity degree of deficiency, (2) the display device is specified based on its spectral emission characteristics, and (3) the color compensation is performed based on the information above.
The basic idea behind color compensation stems from the simulation of anomalous color vision. Since we know the simulation process that mimics the path of human color perception, it is possible to adapt a color so that the simulated color is the same as the original color. Figure 5 shows a pictorial explanation for (a) the simulation of original color perception for anomalous trichromacy, (b) the color compensation that would provide normal color perception to anomalous trichromacy, and (c) the simulation of the compensated color perception for anomalous trichromacy.
In Figure 5(a), a color in the RGB space is converted into a defective color denoted by (l 1 , m 1 , s 1 ) in the LMS space. The (l 1 , m 1 , s 1 ) is subsequently converted into the normal RGB space. The (r 1 , g 1 , b 1 ) is denoted as the defective RGB color. The anomalous color simulation shown in Figure 5(a) leads to the method of color compensation [2,3]. Color compensation is an inverse operation that offsets color defects due to the abnormality of anomalous trichromacy as shown in (r a , g a , b a ) in Figure 5 RGB to LMS conversion Color compensation trichromacy. The output RGB color (r 2 , g 2 , b 2 ) is expected to be the same as the original one (r, g, b). In theory, as seen in Figure 5(b), anomalous trichromats can perceive the original color as normal ones. The compensated color (referred to as r a , g a , b a ) can be computed as follows: where T anomalous n represents the color conversion matrix of anomalous trichromacy given the deficiency degree of n. The T anomalous n can be calculated by the function of cone models, f : n → Δλ estimated (λ) at the different deficiency type. The function can be estimated by the model of spectral sensitivity of anomalous trichromacy, described in Section 2.2. The estimation can be done by multiplying the model of the spectral sensitivity of the anomalous cones (LMS cones) with the spectral emission function of the display device (RGB phosphor) over all visible wavelengths. Then, the resulting 3 × 3 matrix comprises a color conversion matrix of corresponding anomalous trichromacy.

Experimental conditions
The experiments were conducted under daylight condition with roughly 450 lux in illuminance. The color on the monitor was reproduced with a Matrox G550 graphics card, showing over 1024 × 768 pixels of resolution, over 24 bits of true colors. The graphics card has 16 million colors with reproducibility of true colors up to 32 bits and a resolution of up to 2048 × 1536 at the highest color depths and fastest refresh rate. A CRT monitor (Samsung SyncMaster 950 series) was used to display colors with over 75 Hz of screen refresh rate and approximately 6500 K of color temperature corresponding to day light. The monitor was set to be 90% of contrast and 80% of brightness in a dark room without any direct ray of sunlight. The subjects performed the test at 60 cm (arms length) away from the monitor screen.
The monitor was calibrated by a popular calibrator, the X-Rite Monaco Optix Pro [14]. Through the calibration, correction was made of brightness, contrast, color temperature, and gamma of the monitor so that the monitor could display the appropriate colors. To evaluate the monitor calibration results, we measured the calibration error between the colors to be displayed and the measured ones about 24 fixed color patches, which were given by the calibrator. Table 3 shows the calibration error for the 24 color patches. The error was measured using the 1976 CIE L * a * b * color difference. It was approximately 0.8983 per patch. In general, a color difference of 1 dE of error is defined as "just-noticeable difference (JND)" between two colors. Our calibration error results are less than the JND guidelines and thus they must be in the acceptable tolerance range in measurement, that is, little noticed by humans. On the calibrated monitor, the spectral emission functions of R, G, and B phosphors were measured by a spectroradiometer, model Minolta CS-1000.   We also verified whether the compensated colors are always within the gamut of the monitor. Table 4 shows the comparison of CIE chromaticities for the primaries and standard illuminant. Table 4  degree, respectively. Tables 4(e) and 4(f) show the measured compensated-primaries for deuteranomaly of 0.1 degree and that for deuteranomaly of 0.9 degree, respectively. According to the values described in Table 4, the compensated colors show that they are always within the monitor gamut. Figure 6 shows monitor gamut with the compensated colors depicted in the chromaticity diagram: Figure 6(a) is for monitor gamut of ITU-R BT.709 reference primaries and measured primaries, Figure 6(b) is for monitor gamut of measured primaries and measured compensated primaries for protanomaly of 0.1 and 0.9 degrees, and Figure 6(c) is for monitor gamut of measured primaries and measured compensated primaries for deuteranomaly of 0.1 and 0.9 degrees.

Subjects
The experiments were performed in two study centers: one was the University of California, Irvine (UCI) in the USA and the other was Seoul National University Hospital (SNUH) in South Korea. Each center performed the experiments under the same conditions except that the monitor color temperature was approximately 6500 K at UCI and 9000 K at SNUH. The total number of subjects with anomalous trichromacy was 107. All the anomalous subjects were X-chromosomelinked anomalous trichromats, namely protanomalous or deuteranomalous trichromats. UCI collected 92 subjects, who were screened by two pseudoisochromatic tests: HRR and Ishihara tests. Among Seungji Yang et al.    these subjects, the number of inconsistencies was 15, where "inconsistency" means that the TES with color compensation is not much affected for all deficiency degrees. About 10% of the subjects had a little benefit from the experimental color compensation. The number of the normals was 5, where "normal" means that the TES without color compensation is less than "50". The number of the dichromats was 12, where dichromat means that the TES without color compensation is more than 200. Thus, the number of the anomalous trichromat subjects in this experiment was 60, after excluding subjects with inconsistency, normals, and dichromat cases. SNUH collected 65 subjects, who were also screened with HRR and Ishihara tests. Of them, the number of subjects with inconsistency, normals, and dichromats were 6, 2, and 10, respectively. Therefore, the number of anomalous trichromat subjects was 47.

Reduction of error score after color compensation
If the proposed color compensation method is useful to enhance color perception of anomalous trichromats, it will be true that the TES of the CHT using color compensation (referred to as c-CHT) should be lower than that of the CHT using noncompensated color; they may even be under the minimum value for TES for anomalous trichromacy. In these experiments, both CHT and c-CHT were performed on the same subject. The c-CHT was conducted with 5 deficiency degrees of 0.1, 0.3, 0.5, 0.7, and 0.9. The color of the caps in the FM-100H was compensated according to deficiency type and deficiency degree.
The experiment was to find whether or not the TES of the CHT is reduced after color compensation. Thus, the usefulness of the proposed color compensation was tested. Figure 7 shows the TES distribution of the CHT for all subjects. Figure 7(a) shows the TES distribution before color compensation of the subjects in the UCI study, where the mean TES is 142.9. Figure 7(b) shows that after color compensation, the mean TES is 88.8. Figure 7(c) shows the TES distribution before color compensation of subjects in the SNUH study, where the mean TES is 149.8. Figure 7(d) shows that after color compensation, the mean TES is 83.8. In both cases, the TES of the CHT is significantly reduced after color compensation.
To have statistical verification, we performed a test where the null hypothesis is defined as H 0 and an alternative hypothesis as H a . In the hypothesis test, it was assumed that the proposed method would be effective if the mean of the TES of the c-CHT is smaller than the TES limit that defines the subjects as normal trichromats. The null hypothesis is H 0 = μ ≥ μ 0 , where μ is the mean of the sample TES, and μ 0 is the TES limit for normal color vision. The alternative hypothesis is H a = μ < μ 0 . μ 0 was set to 100 for the clinical threshold. To verify this, the one sample t-test was carried out. It tested whether there was sufficient evidence to reject the null hypothesis.
The probability to reject the null hypothesis is shown in Table 5. In results from UCI and SNUH, the P-value is .023 and .006, respectively. Thus, there is sufficient evidence to show that the TES of the c-CHT is reduced to below the TES limit for the normal trichromat, meaning that the color compensation would be helpful to enhance color perception of anomalous trichromats.

Quantitative relationship of color compensation with the CHT
In the experiment above, we verified that the proposed color compensation is useful to enhance the color perception for anomalous trichromats. However, how will the colors be compensated for different types and severity degrees of anomalous trichromacy? This question needs to be resolved. So the following experiment aimed at finding the relationship between color compensation and the deficiency degree standardized by the MPEG-21 multimedia framework.
In the experiment, subjects were examined on both the CHT and color-compensated CHT, called c-CHT. The c-CHT allows producing colors compensated by the proposed scheme. The color compensation is differently performed according to the deficiency degree that is quantified in 10 steps of anomalous cone variations. The deficiency degree is estimated from TES values of the c-CHT and the CHT. Finally, the relationship of the TES in the CHT with the deficiency degree is obtained.  As mentioned in Section 3, the deficiency degree is linearly proportional to the color difference. In these experiments, we assumed that the deficiency degree has a linear relationship with the TES of the CHT. Based on the linear relationship, we predict the true deficiency degree as where TES normal , the maximum total error score allowable to be defined as normal color vision, is 100. This must be obtained by clinical experiments. Clinical testing with real subjects produces some TES samples that may be against the major statistical tendency; this kind of inconsistent sample is called a control sample and may be eliminated from statistical analysis. So in order to more reliably estimate the deficiency degree with the TES samples, we chose several useful TES samples, called candidate (d c ), from an entire set of TES samples. The TES samples that considered to be candidates should be less than a certain TES value (TES normal + ΔTES) after color compensation, where ΔTES is an empirical tolerance value. The final deficiency degree (d(TES)), given the TES in the CHT, is obtained by minimizing the root squared error between the d c and the d. So the estimated deficiency can be written as (4) Figure 8 shows the relationship between the estimated deficiency degree and the TES of the CHT. The upper and lower bounds (mean ± 2 × standard deviaion) about the TES in the CHT are shown for the estimated deficiency degrees. As is seen, data from both UCI and SNUH show a consistently linear relationship (see Figures 8(a) and 8(b)).
A linear equation is modeled using linear regression to estimate the relationship between the TES of the CHT and the deficiency degree. Figure 9 shows the linear equation of estimated deficiency degree versus the TES of the CHT. From the results of UCI, the deficiency degree (d) is (0.005708 × TES − 0.315700). From the results of the SNUH, the deficiency degree (d) is (0.004952 × TES − 0.194994). The two slopes are very similar to each other. From the results of the combination of the SNUH and UCI subjects, the deficiency degree (d) is (0.005336 × TES − 0.258059).
The confidence of the linear equation is measured by using the adjusted R 2 , which is called the coefficient of determination that describes reliability of the regression analysis, and the mean square error (MSE). The adjusted R 2 and MSE of the UCI results are 0.67665 and 0.14806, respectively, while the adjusted R 2 and MSE of the SNUH results are 0.64888 and 0.15670, respectively. The adjusted R 2 and MSE of the results of integration of the SNUH and UCI subjects are 0.66648 and 0.15129, respectively. The linear equation can be regarded as being confident with a large adjusted R 2 and a small MSE. For all cases, the adjusted R 2 is large enough to say that the deficiency degree is linearly related with the TES of the CHT. Figure 10 shows a comparison between the linear function obtained in the experiment and the linear function modeled using the color difference of color compensation mentioned in Section 3. As seen in the results, the regression result is similar to the cone models for every deficiency degree. Note that the vertical axis is not the same; Figure 10(a) is TES, and Figure 10(b) is the color difference.
We also performed the experiment of color compensation with the linear function of deficiency degree that we predicted in Figure 10(a). For this, 15 subjects were additionally collected in SNUH and tested with the CHT. Figure 11 shows the results of the TES before and after color compensation. The deficiency degree is estimated by the linear equation of the SNUH. We see that the TES after color compensation is 92.93, while the TES before color compensation is 139.46. Thus, we can conclude that the TES for all the subjects was reduced near the TES limit for normal trichromats, meaning that the color compensation would work well to improve defective color perception of anomalous trichromats.

CONCLUSIONS
In this paper, we have proposed a novel color compensation method for anomalous trichromats. The detailed contributions of this paper are as follows.
(1) We have modeled anomalous cones (protanomalous L cone and deuteranomalous M cone) according to the deficiency degree standardized by the MPEG-21. The correctness of our approach has been verified by clinical experiments. (2) We have developed a novel method for simulating color perception of anomalous trichromat according to deficiency degree, which aims at interpret the path of human color perception. (3) We have developed a novel color compensation scheme that allows for an improved color perception by anomalous trichromats. The efficiency of the proposed method has been verified by clinical observations. (4) We have discovered a linear relationship between a computerized color vision test, called the CHT, and the deficiency degree. This relationship allows optimizing color compensation for individual defects. This has been also verified by clinical experiments.
In future research, we will perform the color compensation method using the anomaloscope technique that allows for a more precise diagnosis of CVD.