Study of chromatic adaptation using memory color matches , Part I : neutral illuminants

Twelve corresponding color data sets have been obtained using the long-term memory colors of familiar objects as target stimuli. Data were collected for familiar objects with neutral, red, yellow, green and blue hues under 4 approximately neutral illumination conditions on or near the blackbody locus. The advantages of the memory color matching method are discussed in light of other more traditional asymmetric matching techniques. Results were compared to eight corresponding color data sets available in literature. The corresponding color data was used to test several linear (von Kries, RLAB, etc.) and nonlinear (Hunt & Nayatani) chromatic adaptation transforms (CAT). It was found that a simple two-step von Kries, whereby the degree of adaptation D is optimized to minimize the DEu’v’ prediction errors, outperformed all other tested models for both memory color and literature corresponding color sets, whereby prediction errors were lower for the memory color sets. The predictive errors were substantially smaller than the standard uncertainty on the average observer and were comparable to what are considered just-noticeabledifferences in the CIE u’v’ chromaticity diagram, supporting the use of memory color based internal references to study chromatic adaptation mechanisms. © 2017 Optical Society of America OCIS codes: (330.0330) Vision, color, and visual optics; (330.4060) Vision modeling; (330.7320) Vision adaptation; (330.1720) Color vision; (330.5020) Perception psychology; (330.5510) Psychophysics. References and links 1. CIE160–2004, “A review of chromatic adaptation transforms” (CIE, Vienna, 2004). 2. N. Moroney, M. D. Fairchild, R. W. G. Hunt, C. Li, M. R. Luo, and T. Newman, “The CIECAM02 color appearance model,” in IS&T/SID Tenth Color Imaging Conference (2002), p. 23. 3. H. E. Ives, “The relation between the color of the illuminant and the color of the illuminated object,” Transactions of the Illuminating Engineering Society 7, 11 (1912). 4. H. Helson, “Some factors and implications of color constancy*,” J. Opt. Soc. Am. 33(10), 555–567 (1943). 5. D. H. Foster, “Color constancy,” Vision Res. 51(7), 674–700 (2011). 6. I. Kuriki, “The loci of achromatic points in a real environment under various illuminant chromaticities,” Vision Res. 46(19), 3055–3066 (2006). 7. L. M. Hurvich and D. Jameson, “A psychophysical study of white. I. Neutral adaptation,” J. Opt. Soc. Am. 41(8), 521–527 (1951). 8. K. A. G. Smet, G. Deconinck, and P. Hanselaer, “Chromaticity of unique white in illumination mode,” Opt. Express 23(10), 12488–12495 (2015). 9. A. G. Kevin, D. Geert, and H. Peter, “Chromaticity of unique white in object mode,” Opt. Express 22(21), 25830–25841 (2014). 10. M. D. Fairchild, “Formulation and testing of an incomplete-chromatic-adaptation model,” Color Res. Appl. 16(4), 243–250 (1991). 11. R. W. G. Hunt, “Revised colour-appearance model for related and unrelated colours,” Color Res. Appl. 16(3), 146–165 (1991). 12. M. D. Fairchild, Color Appearance Models, 2 ed. (John Wiley & Sons, 2005), p. 408. 13. K. M. Lam, “Metamerism and colour constancy. Ph.D. thesis,” (University of Bradford, 1985). 14. G. D. Finlayson and S. Susstrunk, “Spectral sharpening and the Bradford transform,” in Proc. of the Color Imaging Symposium 2000, 236 −242. 15. C. Li, M. R. Luo, B. Rigg, and R. W. G. Hunt, “CMC 2000 chromatic adaptation transform: CMCCAT2000,” Color Res. Appl. 27(1), 49–58 (2002). Vol. 25, No. 7 | 3 Apr 2017 | OPTICS EXPRESS 7732 #282724 https://doi.org/10.1364/OE.25.007732 Journal © 2017 Received 13 Dec 2016; revised 22 Feb 2017; accepted 19 Mar 2017; published 27 Mar 2017 16. G. D. Finlayson, M. S. Drew, and B. V. Funt, “Spectral sharpening: sensor transformations for improved color constancy,” J. Opt. Soc. Am. A 11(5), 1553–1563 (1994). 17. F. A. Dunn, M. J. Lankheet, and F. Rieke, “Light adaptation in cone vision involves switching between receptor and post-receptor sites,” Nature 449(7162), 603–606 (2007). 18. P. B. Delahunt and D. H. Brainard, “Control of chromatic adaptation: signals from separate cone classes interact,” Vision Res. 40(21), 2885–2903 (2000). 19. M. Kamermans, D. A. Kraaij, and H. Spekreijse, “The cone/horizontal cell network: a possible site for color constancy,” Vis. Neurosci. 15(05), 787–797 (1998). 20. D. H. Brainard and B. A. Wandell, “Asymmetric color matching: how color appearance depends on the illuminant,” J. Opt. Soc. Am. A 9(9), 1433–1448 (1992). 21. D. Jameson and L. M. Hurvich, “Color adaptation: sensitivity control, contrast, after-images,” in Handbook of Sensory Physiology, L. M. H. D. Jameson, ed. (Springer, 1972), pp. 568–581. 22. S. K. Shevell, “The dual role of chromatic backgrounds in color perception,” Vision Res. 18(12), 1649–1661 (1978). 23. J. Walraven, “Discounting the background--the missing link in the explanation of chromatic induction,” Vision Res. 16(3), 289–295 (1976). 24. J. Walraven, C. Enroth-Cugell, D. C. Hood, D. I. A. MacLeod, and J. L. Schnapf, “5 The control of visual sensitivity: receptoral and postreceptoral Processes A2 Spillmann, Lothar,” in Visual Perception, J. S. Werner, ed. (Academic Press, 1990), pp. 53–101. 25. M. M. Hayhoe, “Spatial interactions and models of adaptation,” Vision Res. 30(6), 957–965 (1990). 26. M. D. Fairchild, Color Appearance Models, 3 ed. (John Wiley & Sons, 2013), p. 450. 27. W. S. Geisler, “Mechanisms of visual sensitivity: backgrounds and early dark adaptation,” Vision Res. 23(12), 1423–1432 (1983). 28. P. T. Kortum and W. S. Geisler, “Adaptation mechanisms in spatial vision--II. Flash thresholds and background adaptation,” Vision Res. 35(11), 1595–1609 (1995). 29. R. L. De Valois, “Neural coding of color,” in The Visual Neurosciences, L. M. Chalupa and J. S. Werner, eds. (MIT, 2004), p. 1808. 30. J. M. Valeton, “Photoreceptor light adaptation models: an evaluation,” Vision Res. 23(12), 1549–1554 (1983). 31. M. A. Webster and J. D. Mollon, “Colour constancy influenced by contrast adaptation,” Nature 373(6516), 694–698 (1995). 32. K.-H. Bäuml, “Simultaneous color constancy: how surface color perception varies with the illuminant,” Vision Res. 39(8), 1531–1550 (1999). 33. M. P. Lucassen and J. Walraven, “Quantifying color constancy: evidence for nonlinear processing of conespecific contrast,” Vision Res. 33(5-6), 739–757 (1993). 34. J. S. Werner and J. Walraven, “Effect of chromatic adaptation on the achromatic locus: the role of contrast, luminance and background color,” Vision Res. 22(8), 929–943 (1982). 35. M. D’Zmura and P. Lennie, “Mechanisms of color constancy,” J. Opt. Soc. Am. A 3(10), 1662–1672 (1986). 36. J. A. Worthey and M. H. Brill, “Heuristic analysis of von Kries color constancy,” J. Opt. Soc. Am. A 3(10), 1708–1712 (1986). 37. V. Ekroll, F. Faul, and G. Wendt, “The strengths of simultaneous colour contrast and the gamut expansion effect correlate across observers: evidence for a common mechanism,” Vision Res. 51(3), 311–322 (2011). 38. D. L. MacAdam, “Chromatic Adaptation. II. Nonlinear hypothesis*,” J. Opt. Soc. Am. 53(12), 1441–1445 (1963). 39. Y. Nayatani, K. Takahama, H. Sobagaki, and K. Hashimoto, “Color-appearance model and chromaticadaptation transform,” Color Res. Appl. 15(4), 210–221 (1990). 40. Y. Nayatani, T. Yano, and M. Ihara, “Analyses of methods for predicting corresponding colors of LUTCHI data,” Color Res. Appl. 27(5), 335–348 (2002). 41. L. Mori, H. Sobagaki, H. Komatsubara, and K. Ikeda, “Field trials on CIE chromatic adaptation formula,” in CIE 22nd Session (Melbourne, 1991), pp. 55–58. 42. H. Helson, D. B. Judd, and M. H. Warren, “Object-color changes from daylight to incandescent filament illumination,” J. Illum. Eng. 47, 13 (1952). 43. M. R. Luo, A. A. Clarke, P. A. Rhodes, A. Schappo, S. A. R. Scrivener, and C. J. Tait, “Quantifying color appearance.1.LUTCHI color appearance data,” Color Res. Appl. 16(3), 166–180 (1991). 44. W. G. Kuo, M. R. Luo, and H. E. Bez, “Various chromatic-adaptation transformations tested using new color appearance data in textiles,” Color Res. Appl. 20(5), 313–327 (1995). 45. E. J. Breneman, “Corresponding chromaticities for different states of adaptation to complex visual fields,” J. Opt. Soc. Am. A 4(6), 1115–1129 (1987). 46. K. M. Braun and M. D. Fairchild, “Psychophysical generation of matching images for cross-media color production,” in IS&T and SID's 4th Color Imaging Conference: Color Science, Systems, and Applications (IS&T, Springfield, USA, 1996), pp. 214–220. 47. J. J. McCann, S. P. McKee, and T. H. Taylor, “Quantitative studies in retinex theory comparison between theoretical predictions and observer responses to color Mondrian experiments,” Vision Res. 16, 445-& (1976). 48. M. R. Luo and R. W. G. Hunt, “Testing colour appearance models using corresponding-colour and magnitudeestimation data sets,” Color Res. Appl. 23(3), 147–153 (1998). Vol. 25, No. 7 | 3 Apr 2017 | OPTICS EXPRESS 7733


Introduction
Chromatic adaptation refers to the ability of the human visual system to (partially) adapt to the intensity and color of the illumination, producing an approximately color constant appearance of objects across changes in illumination.
Models that predict the adaptive shift due to chromatic changes in lighting/viewing conditions are referred to as Chromatic Adaptation Transforms (CATs) and are an important part of color appearance models (CAMs).As an example, the CAT02 transform [1] is embedded in CIECAM02 [2].The development of CATs is typically based on one or more sets of corresponding colors (CC), which are stimuli that appear equal in color under different illumination conditions.Corresponding colors are commonly derived using asymmetric matching where observers have to match the color appearance of a stimulus under the test illuminant with that under a reference illuminant.
In this paper, corresponding colors were obtained using a novel experimental method that involves the long-term memory colors of familiar objects as internal references.Twelve corresponding color sets were determined for 4 'neutral' illumination conditions at one fixed adapting luminance of 760 cd/m 2 (approx.2600 lux).Data and results on more colored illumination will be presented in part II of this paper.Each CC set contains corresponding colors for 5 familiar objects of different hues (neutral, red, yellow, green and blue).
The paper starts with an overview of chromatic adaptation, its modeling and the typical asymmetric matching techniques used to determine corresponding color sets.It then continues with a discussion of the memory color matching approach and the experimental setup.After an analysis of the experimental results and the derivation of the new corresponding color data set, the latter was used to test the performance of various chromatic adaptation transform models from literature.

The von Kries chromatic adaptation transform
Chromatic adaptation is traditionally modelled in first approximation by the von Kries coefficient rule (Eq.( 1)) which states that the cone sensitivity functions (or resulting cone excitations) are independently scaled, i.e. each cone channel has a separate gain (multiplicative) control that does not depend on the other channels, but only on the adaptation state: L, M and S are the Long, Medium and Short wavelength cone excitations; k L,M,S are the cone gain controls; and subscripts 0 and a denote respectively baseline and adapted cone signals.
Von Kries did not mention how exactly the gain control coefficients k L,M,S are to be calculated.However, two common approaches, respectively referred to as the von Kries -Ives and the von Kries -Helson model, effectively renormalize the adapted cone responses to an illuminant-independent baseline state using the illuminant color [3] or the average visual field [4] The k L,M,S gain control factors for the Ives and Helson models are respectively: whereby subscript w and vf resp.refer to the illuminant white point and the visual field.The two approaches are equivalent under the gray world assumption [5], often used in color constancy research, which states that in general the average spectral reflectance of the objects and surfaces in the surround are neutral.
To be able to serve as a reference for more or less chromatic illumination conditions under incomplete adaptation, the illuminant independent baseline state is typically assumed to correspond to the neutral (achromatic) point, i.e. the stimulus that looks neither red, yellow, green or blue, under dark adapted conditions.The equi-energy-white (EEW) [6,7] is therefore.often adopted as the intrinsic white point in many chromatic adaptation transforms and color appearance models.However, it should be noted that the observer variability under dark adapted conditions is quite large and several other unique white (neutral) chromaticities have been reported.For an overview, see [8,9].
Finally, the transformation from adaptive condition A to adaptive condition B can then be determined as follows:

Degree of adaptation
The transform as formulated in Eq. ( 1) and Eq. ( 2), assumes the degree of adaption to be complete.However, the adaptation degree, D, is dependent on both the luminance of the adapting field as well as its chromaticity.However, the latter is often ignored in many CAT adopted in color and imaging science.For example, the CAT02 chromatic adaptation transform calculates D as follows [2]: with La the luminance of the adapting field and F a factor that is respectively 0.8, 0.9 and 1.0 for a dark, dim and average surround.The degree of adaptation D is included in the CAT model as follows: (1 ) Fairchild [10] introduced a function for the degree of adaptation, based on the Hunt model [11], in the RLAB color appearance model that includes the effect of both the luminance and the chromaticity of the adaptive condition [12].The von Kries multiplicative gain control factor a L,M,S in the RLAB model are given by: [ ] ( ) with X n the cone responses for the adapting stimulus and p X the proportion of complete von Kries adaptation.Note that in this case D represents an additional factor, often referred to as the degree of discounting the illuminant, that regulates the effective degree of adaptation.Note that the term discounting-the-illuminant is typically used when referring to cognitive (cortical) processes that allow the visual system to further adapt to or adjust for the color and intensity of the illumination, whereas chromatic adaptation is usually limited to early (often retinal) sensory processing mechanisms (either receptoral or post-receptoral).
Depending on the values of p X and D, the effective degree of adaptation may or may not range between completely no and completely full adaptation.The proportion of complete von Kries adaptation p X is given by: with de luminance of the adapting stimulus in cd/m 2 , and x E the modified fundamental chromaticity coordinates of the adapting stimulus with respect to the equal energy white E: ( ) Note that the L,M,S responses are to be calculated from the relative tristimulus values.

Chromatic adaptation sensors
Although von Kries intended the coefficient rule to apply to cone signals, other, often sharper, 'sensors' (e.g.Bradford or BFD [13], Sharp [14], CAT02 [2],CMCCAT2000 [15],…) have been used as well, because of their ability to better account for corresponding color data.It has been shown mathematically that the von Kries coefficient rule becomes appropriate as the sensors become sharper [16].Sharpened sensors also appear to be psychophysically relevant in other areas (for a short overview see [14]) as well as physiologically plausible.While gain control of the cone channels can be explained physiologically at higher light levels by bleaching (depletion) of the photosensitive pigments in the cones, at lower levels there is insufficient bleaching to account for the adaptation.Dunn et al. [17] found that the site of adaptation switches from the receptor cells to the postreceptoral neural circuitry as light intensity drops.Gain control at level of the horizontal, bipolar and ganglion cells could lead to a sharpening of the effective sensors [15].In addition, such non-receptor gain control also helps account for the observed spatially low-pass characteristics of chromatic adaptation [12].

Dependent multiplicative gain control and subtractive adaptation
Application of independent multiplicative sensor regulation (corresponding to a diagonal matrix multiplication as in Eq. ( 1)) is often referred to as a strong von Kries model, while the use of a model whereby the sensor gains are not independent, such as in a linear or affine model, is referred to as a weak von Kries model.Delahunt & Brainard [18] indeed found psychophysical evidence for interactions between the various cone channels.Kamermans et al. [19], found physiological support for cone interactions in goldfish through a spectrally and spatially broad feedback of the horizontal cells into the photoreceptor synapses of all cones, thereby effectively encoding color constancy into the retina.Affine models [20], that include a matrix multiplication and a vector addition (subtraction) are a generalization of the two-stage adaptive process as envisioned by [21] and are an attempt to better predict the changes in color appearance due to changes in adaptive conditions.Psychophysical evidence for subtractive mechanisms in addition to multiplicative ones has been found by for example Shevell [22] and Walraven [23].Subtractive adjustments of the post-receptoral pathways serve to discount or filter out the background signal [24] and are thought to be located at center-surround sites in the retina [25].

Nonlinear response compression
Physiological models for adaptation based on both multiplicative and subtractive mechanisms can be made compatible with models that use only a multiplicative gain control by assuming that the subtractive adjustments take place after a nonlinear compression: for a logarithmic compression, a subtractive change after compression is equivalent with a multiplicative change before compression [26].However, models with a subtractive adjustment prior to a nonlinear compression have been found to fit psychophysical flashprobe data well [27,28].Non-linear compression occurs at several sites in the visual pathway.Although the phototransduction in the cones is quite linear, several non-linear processes in the cone's synaptic region limit the dynamic range of the system from about one billion to 1 to at most several hundred to 1 [29].Post-receptoral cells, such as the bipolar and the ganglion cells have, like most neurons, a limited response range that is often modelled by a nonlinear compression.One of the most commonly used models for response compression (or also light adaptation), in addition to a simple logarithmic function, is the Michaelis-Menten function [30]: with R the response, R max the maximum possible response, I the signal intensity, σ the intensity value that produces half the response of R max .and n a factor that controls the steepness of the response function.
Although present in the visual system, nonlinearities are often ignored in practical chromatic adaptation transforms as a simple von Kries CAT has been found to work quite well as a first order approximation [20,31,32].On the other hand, it should be bore in mind that small systematic deviations from linearity as implied by a von Kries like transformation have been found [6,33] and that it cannot provide a full account for color constancy under different illuminations due to other competing effects such as contrast [31,[34][35][36][37].

One-step and two-step von Kries type chromatic adaptation transforms
In practice, corresponding colors between two adaptive conditions A and B are typically calculated using a one-step von Kries transform of the form (see also Eq. ( 5) and Fig. 1): with D A,B the degree of adaptation, X A and X' B respectively the adapted sensor responses for a stimulus in illumination condition A and the corresponding adapted sensor responses under condition B, and A B → Λ the diagonal matrix composed of the gain controls of each sensor channel: whereby X wA and X wB the sensor responses for the white point under adaptive condition A and B respectively.However, the interpretation of the degree of adaptation D A,B in one-step transforms is quite unclear: does it correspond to the adaptation level to condition A or B, or to some mixed condition AB?The difficulty arises, because the one-step CAT ignores that each transform from each condition to the illuminant independent baseline state (see Eqs. ( 1)-( 3)) is subject to its own degree of adaptation.Another more appropriate approach is therefore to calculate corresponding colors using a two-step chromatic adaptation transform.A two-step CAT first explicitly transforms the color under illumination condition A to a baseline state X 0 (which in the CAT02 transform is the EEW) and then further transforms from there to illumination condition B (see Fig. 1).Each step has its own degree of adaptation.Using analogous notations, a two-step CAT can be defined as: Note that a two-step transform is mathematically equivalent to a one-step model for ,0 ,0 ,

Corresponding colors
Several data sets of corresponding colors-colors that match under different adaptive conditions-can be found in literature and have been used in the development of various CATs.They have been collected through a number of different techniques [1], each with its own advantages and advantages.To situate the technique presented in this paper, i.e. memory color matching, a short overview of commonly adopted methods is given.The focus is on asymmetric matching methods, as the method of magnitude estimation requires a complicated analytical or empirical procedure for data analysis to derive corresponding colors [40].

Successive, simultaneous, haploscopic and memory matching
In asymmetric matching experiments, an observer is presented with a stimulus under test illumination conditions and is asked to make or select a match under some reference illumination.Several experimental techniques exist.
In a successive matching experiment, test and reference conditions are presented one after the other.This method relies therefore heavily on the short term memory of the test subject [49] and typically requires frequent switching back and forth between the two illumination conditions, a process which can be time-consuming as the observer requires readaptation for each switch.
Simultaneous matching is easier as both conditions are presented simultaneously to the observer.However, the adaptation state is affected by eye-movements across the two halves of the scene [49].This is resolved by a haploscopic matching setup, in which each eye is presented with either the test or reference conditions.However, the restricted eye movements result in less natural viewing conditions.In addition, the interaction of the signals of both eyes at postretinal processing stages might have an effect on the state of adaptation [50,51].
Another method that has been used that affords more complete adaptation and free eye movements and that represents more natural viewing conditions is memory matching.Test subjects are first trained in a color order system, such as the Munsell system, until they can reliably evaluate color in terms of its scales (value, chroma and lightness).They are then presented with several stimuli under different illuminants and are asked to describe its color appearance.This method avoids switching back and forth between a test and reference illuminant.However, obtaining such highly trained observers requires a substantial training period [1].

Memory color matching (MCM)
An obvious solution-although surprisingly un(der)used within the context of corresponding color and adaptation research-to the problem of regular memory matching is to have people match colors they already know well, such as the colors associated with familiar objects in long-term memory, so called memory colors.Note that memory color and color memory are two distinct concepts.While the former refers to the colors associated with familiar objects, the latter refers to the general ability to remember colors.
The Memory Color Matching concept is straightforward: an observer is presented with a familiar object in an atypical color under each adaptive condition and is asked to adjust its color appearance until it matches the internal memory color.Because the color settings correspond to the same color appearance under different adaptive conditions, their chromaticity and luminance values form a set of corresponding colors.A total of N(N-1) corresponding color sets can be derived from just N matching experiments.Note that in this study, observers were explicitly instructed to make color appearance matches and not surface matches [52].The two tend to generate different sets of corresponding colors.Appearance matches are mainly determined by sensory or adaptational mechanisms (what is typically referred to as chromatic adaptation), while surface matches involve perceptualcomputational and inferential mechanisms (i.e.relational color constancy) [5].
Memory colors have been used successfully in the past as an internal reference to study the color rendering of white light sources [53,54] or image color quality [55][56][57].MCM is analogous to other approaches based on internal references often used in color constancy research, such as achromatic matching and unique hue setting.However, these approaches only allow for respectively one and four internal references whereby shifts in stimulus lightness and/or chroma are ignored.To increase the number of potential internal references, Roca-Vila, et al. [58] have recently demonstrated the feasibility of using focal colors in the study of color constancy.The adoption of memory colors as internal references further extends this paradigm to an even wider range of potential target stimuli.
Compared to some of the more traditional approaches for obtaining corresponding color data sets, memory color matching has several advantages: • A substantial increase in the number of possible internal references.
• Matches are expected to be more accurate compared to short-term 'learned' memory matches [49] (cfr.external references from e.g. the Munsell atlas).
• It is less time consuming than asymmetric matching as no paired presentations are required.
• It has many of the benefits of successive matching: unrestricted eye movements resulting in a more natural viewing conditions and good control of the observer's adaptive state which can be affected by the presence of the reference field in simultaneous [49] or haploscopic matching (Barbur and Spang, 2008, Vimal and Shevell, 1987).
• No need for switching back-and-forth between test and reference conditions eliminating numerous observer re-adaptation.
• It requires no extensive training in a color order system (cfr.magnitude estimation) [1].Possible disadvantages are: • Memory color matches might be less sensitive to changes in viewing conditions and therefore be less representative [59], but see [60].
• There aren't familiar objects for any desired chromaticity (Blue or magenta familiar objects are difficult to find [61]).

Experimental setup
Memory colors were obtained under 4 different 'neutral' adapting illuminants for five objects: a gray cube, a green apple, a ripe lemon, a ripe tomato and a blue smurf.Objects were selected to sample different parts of the hue circle.Note that although gray is not really a memory color, it is also an internal reference 'color' and has often been used in color constancy research (cfr.achromatic settings).In this paper, 'gray' is therefore grouped together with actual memory colors for brevity.
Based on the results in Smet et al. [8,9], the correlated color temperature (CCT) of the 'neutral' illumination was restricted to range between 3500 K and 10000 K, while the Duv, the distance from the blackbody locus in the 1960 uniform color space (UCS) of the International Commission on Illumination (CIE), was kept below 0.015.Outside this region the degree of neutrality drops substantially [8,9], indicative of a decreasing adaptation to more colored stimuli, as has also been reported by others [6,63].The following illumination conditions were chosen: a neutral [N] obtained by Smet et al. [8,9], EEW, D65, and a Planckian radiator of 4000 K.
As this paper primarily focuses on exploring the feasibility of the memory color matching method, the adaptive field luminance of the spectrally neutral background was limited to a single value, approximately 760 cd/m 2 (~2600 lux).The background scene, illustrated in Fig. 2(a), had approximately a 50° field of view and was populated with various spectrally neutral (gray) 3D objects to enhance scene realism by providing depth, parity and illumination cues, as they have been reported to color constancy and chromatic adaptation [5].However, colored objects were not included as to not bias the overall background chromaticity [64] compared to the gray world often assumed in color constancy research as a possible cue to estimate the illuminant.The background illumination was provided by a calibrated data projector.The use of a data projector allows for the independent adjustment of stimulus area and background area, thereby separating the adaptive shift from the illuminant colorimetric shift typically present in asymmetric matching studies using reflective samples.This avoids additional bias due to the color rendition of the test and reference Independent control of the color of the background illumination and the color of test objects of various shapes and sizes could be easily achieved by the use of a mirror underneath the test object, a careful relative positioning of the background, projector and observer and by the projection of a simple twocolor image.A side view schematic of the experimental setup is given in Fig. 2

(b).
In the experiments, test object chromaticity and luminance could be adjusted by the observers using the arrow keys on a keyboard while navigating in the CIE u'v' diagram.
Each key press only changed the RGB output of the data projector pixels corresponding to the 3D object's projection on the imaging plane, resulting in only a change in the object's color.Each memory color setting was spectrally recorded using a telespectroradiometer (Bentham TEL301 telescope coupled to an Ocean Optics QE65 Pro spectrometer).Tristimulus and chromaticity coordinates are calculated from the spectral radiance measurements using the CIE 1964 (10°) observer.

Experimental procedure
Observers were first allowed to dark adapt for 5 minutes, during which the experimental procedure was explained to them, as well as given detailed instructions on the use of the keyboard to change the color appearance of the presented familiar object until it matches their memory color.
For each illumination condition and object, a total of 4 memory color settings were made by an observer, whereby the initial chromaticity of the test objects at the start of each match was varied to minimize starting bias.Starting chromaticity was selected to be equally distributed along the hue circle centered at the familiar object chromaticity when illuminated by an equal-energy spectrum, but whereby the radius of the hue circle was varied for each selected hue angle to be just within the projector gamut.At the start of each new illumination condition, the observer was requested to wait for approximately 15 seconds before starting to make his memory color match to ensure a substantial adaptation has taken place.It is known that 50% of the steady-state adaptation level is reached within the first few seconds [65].By the time an observer has finished his match, which takes at least one minute, approximately 90% or more of the steady-state adaptation level will have been reached.To minimize the potential for a mixed adaptation state between the familiar object color and the color of the background adapting field and maximize adaptation to the latter observers were instructed not to stare at the presented familiar object while making their matches.
After an observer had made a satisfactory match, he/she was requested to rate the quality of his match on a 0-10 scale by pressing one of the F1-F11 keys on the keyboard.The match was then spectrally recorded and the illumination was changed to a new condition.
Note that the memory color data for the 4 neutral illuminations conditions were collected together with that for 9 more colored illuminants, which will be reported on in a part II of this paper [66].To avoid observer fatigue due to the large number of required memory color matches (52 = 13 illumination conditions x 4 starting points) on a single object, data was therefore obtained in two sessions on separate days.In each session and for each observer, the presentation order of the illumination conditions and starting points was randomized to minimize order bias.

Observers
Twenty-three color-normal observers, as tested by the Ishihara-24-plate test, participated in the experiments.The average age of the observers was 33 years with a standard deviation of 10 years.For each object and illumination condition, memory colors were determined by ten observers (5 male and 5 female).A prerequisite for each observer was familiarity with the prototypical color of the object presented.

Observer variability and average observer uncertainty
Inter-observer memory color variability was estimated by calculating for each individual observer for each object and illuminant condition the u'v' color difference, DE u'v', with the mean memory color of all observers and taking the median (MedCDM, Median color difference with the mean).A median was selected because of the skewed distribution of color differences values due to the lower bound at zero.Intra-observer variability was assessed with regard to an observer's own mean memory color (of the four matches per illumination condition per object).In addition, observer variability was also assessed using the Standardized-Residual-Sum-of-Squares (STRESS) [67].Low values signify good agreement, either within (intra) or between (inter) observers.
The uncertainty (in u'v' units) of the average observer's memory color of an object for each background was estimated as the average length of the major and minor axis of the standard error ellipses around the mean memory color.
Finally, the reliability of the concept of an average observer based on a limited number of individual observers was assessed by an ICC(2,n) Intraclass Correlation Coefficient [68].The ICC ranges from 0 (no agreement) to 1 (perfect agreement).
Note that compared to simultaneous color matching or color discrimination studies interobserver variability is expected to be larger for memory color matching as in the latter the reference color is observer specific.

Derivation of corresponding colors (CC)
Corresponding colors are stimuli that appear similar in color under different illumination (adaptive) conditions.In this study the target colors for matching under each illumination condition were the same (i.e. the object memory colors), therefore, 12 ( = 4(4-1)) sets of corresponding color pairs could be derived by a pair wise combination of the memory color data (i.e. the chromaticity values calculated from the measured spectral data) obtained under each illumination condition.For example, N to D65, D65 to N, N to EEW, EEW to N, etc.

Testing chromatic adaptation transforms (CAT)
The predictive performance of various linear and nonlinear CATs were tested using the new corresponding color data sets.Linear CATs included a one-step (Eq.( 10)) and two-step (Eq.( 12)) von Kries model with Hunt-Pointer-Estevez (HPE) cone primaries [12] and with CAT02 primaries [2], while nonlinear CATs included the one adopted in the Hunt model [11] and the one developed by Nayatani [69].
Model performance was tested by setting the degree of adaptation (or discounting the illuminant) D according to three functions (D CAT02 , D CMCCAT97 and D CMCCAT2000 ) published in literature that take the adapting luminance and surround brightness ('average', 'dim' or 'dark') into account, by setting D to 1 (full adaptation or full discounting-the-illuminant) and by optimizing the value of D such that the median CIE 1976 DE u'v' was minimized for each corresponding color data set.Note that the Nayatani model and Hunt models do not include discounting-the-illuminant factors.However, following RLAB, the Hunt model was extended with a D factor to control the contribution of the chromatic adaptation factors F ργβ .The Hunt model also includes a correction for the Helson-Judd effect whereby achromatic surfaces lighter and darker than the background take on respectively the hue and complementary hue of the illumination.Although probably unimportant in practical viewing conditions [12], the impact of its effect was investigated by adding a "degree of Helson-Judd correction" D HJ to the model that was optimized to minimize DE u'v' or set to either 0 (no HJ correction) or 1 (full HJ correction).The Nayatani model is less easily extended with a degree of discounting-the-illuminant and was therefore kept as is.
Performance was checked by calculating the median color differences between the predicted and experimental corresponding colors.The following color difference formulas were used: DE u'v' (CIE 1976 u'v' chromaticity diagram), DE*ab (CIELAB) and DE00 (CIEDE2000) [70].

Observer variability
The median intra-and inter-observer MedCDM values calculated across all objects and illumination conditions are respectively 0.0122 and 0.0162 u'v' units.Average observer standard uncertainty is 0.0069 u'v' units and STRESS values are respectively 40% and 40%.The STRESS value for intra-observer variability is substantially higher than the 22% reported in [62] for memory color ratings under D65, indicating that observers found matching more difficult than rating or that there is potentially a starting bias for the individual matches.On the other hand, the value for the inter-observer variability was only slightly worse than the reported value of 36%, indicating that averaging the individual matches leads to a similar level of agreement as in a memory color rating experiment.In addition, it also indicates that any potential starting bias is substantially reduced or even completely eliminated in the averaged results.As expected, compared to color discrimination studies where the reference color is the same for each observer, the interobserver variability for the memory color matching in the present study is larger, but only slightly: e.g.STRESS values of 35 [73] or 37 [72] compared to the 40 found for this study.The good to excellent observer agreement-note that individual observer memory colors need not be identical-was also confirmed by the high value of the ICC(2,n) intraclass correlation coefficient (0.89).

Testing chromatic adaptation transforms (CAT)
The prediction errors in terms of DE u'v' , DE* ab and DE00 for the various tested models for the corresponding memory color data sets and those from literature are given in Table 1.
Results are presented for the models' native sensor spaces, while for the von Kries models, results are also presented for the widely used CAT02 sensor space.The CC data of all sets are restricted to the 'neutral' illumination conditions mentioned above (see also Table 1).
Comparing the values for the various models in Table 1A it is clear that a simple von Kries CAT with optimized D values and applying a two-step transform performs the best for all investigated subsets.The results also show that a one-step von Kries CAT performed directly between illumination conditions A and B performs slightly, but significantly (Wilcoxon signed rank tests (WSR), p < 0.05), worse.
Both one-and two-step models also have predictive errors that are substantially smaller than the standard uncertainty on the average observer and that are comparable to what are considered just-noticeable-differences in the CIE u'v' chromaticity diagram, which supports the use of memory color based internal references in asymmetric matching experiments designed to study chromatic adaptation mechanisms.Native sensors DEu'v'(1e-3) 1.6 2.0 10.3 6.5 10.3 6.5 7.5 8.5 7.9 7.9 7.9 11.6 7.9 12.4 DE*ab 1.6 1.9 6.1 3.7 6.1 3.7 4.2 4.9 4.5 4.6 4.5 6.9 4.5 8.1 DE00 0.9 1.0 3. Note that these are the only two models whereby the degree of effective adaption is free to range between completely no adaptation and completely full adaptation.In the other models, D is either completely lacking (e.g. the Nayatani model), is acting on some other function regulating adaptation (e.g. in the RLAB and Hunt models), is completely determined by the adapting luminance and surround viewing conditions or is kept fixed.Obviously the reduced freedom in these models could have a negative impact on their predictive performance.The Akaike Information Criterion (AIC) [73] was used to compare the performance of the models while taking the degrees of freedom available during the optimization into account.The analysis confirmed the better performance of the two-step model compared to the one-step CAT and of both two-and one-step CATs compared to the other models: the likelihoods of the two-step, one step and other models were respectively 0.770, 0.23 and <0.001.The results of the first two models are therefore a good benchmark for the performance that should be achievable under the von Kries assumption.
From Table 1A, it is clear that for the native sensor space of the von Kries transform, HPE, that setting D = 1 results in large prediction errors, similarly for models that include a D-function based on the adapting luminance.The CAT02 sensors result in a small improvement.For both sensor spaces, the largest errors occur for D CMCCAT2000 which has a value of 1 for the adapting luminance (760 cd/m 2 ) and dark surround of this study; for both the D CAT02 and D CMCCAT97 model D = 0.8, resulting is smaller prediction errors.Although does better than the von Kries CAT with D = 1 and HPE sensors (and comparable performance when using CAT02 sensors), it provides a worse prediction than the von Kries CATs with D CAT02 and D CMCCAT97 .Here also, the degree of adaptation is largely determined by the adapting luminance, with chromaticity providing only a relatively small contribution (see Eq. ( 7)), especially for the neutral illumination condition under study.The Hunt and Nayatani nonlinear models also perform quite poorly compared to the simple von Kries models with freely optimized D. This is surprising given their very complex structure that tries to include many of the mechanisms involved in human color vision.Part of the poor performance could, at least for the Nayatani model, be attributed to the lack of any discounting of the illuminant (no D factor that could be optimized).However, even then it has a higher prediction error than other CATs where D has been kept fixed (e.g.von Kries with D = 1, RLAB without D contribution and the Hunt model without D or D HJ ).The Hunt model on the other hand, does include such a factor.However, as in RLAB, the effective degree of adaptation might be mainly driven by the high adapting luminance.The results further show that the Helson-Judd effect included in the Hunt model has little impact on predictive performance.
From Table 1B, it can be observed that the Hunt models and RLAB model with optimized D had the same level of performance for the literature sets as the one-step and two-step von Kries models with optimized D and HPE sensors (CAT02 provides slightly higher prediction errors).Of the von Kries models with optimized D, the two-step von Kries model was again the better model (WSR, p < 0.05).The other models, without optimized D, performed substantially poorer for the HPE sensors, while performance was comparable when using the CAT02 sensors, except for perhaps in the case D = 1.
Comparing the results for the memory color based and literature corresponding color sets it is clear from the values in Tables 1A and 1B that the one-step and two-step von Kries models with free effective adaptation level have substantially lower predictive errors for the corresponding memory color sets than for sets from literature.For the other von Kries models the results indicate a slightly better performance for the memory color sets when using the D CAT02 and D CMCCAT97 functions and a comparable prediction error for models with D values equal to 1.The Hunt without optimized D performed slightly better for memory color sets, while prediction errors were lower for literature sets for the Hunt model with optimized D, the RLAB models and the Nayatani model.
From the above analysis it is clear that a failure to accurately account for effective degree of adaptation can lead to substantial decreases in predictive performance.Such failure could be due to either incompleteness (e.g.no chromaticity dependence) or plain error in the model itself, to noise in the psychophysical data; or to other processes than von Kries adaptation that contribute to object color appearance, such as for example simultaneous contrast.
Given that the corresponding color data, and especially the memory color based data, could be adequately predicted by a simple von Kries chromatic adaptation transform it is reasonable to assume that (independent) gain control of the sensors is the main mechanism determining object color appearance under different illumination conditions in a neutral surround.However, the von Kries models with D set to 1 or with D calculated based on the adapting luminance according to either the D CAT02 , D CMCCAT97 or D CMCCAT2000 functions performed sub-optimally for the corresponding memory color sets, indicating they overestimated the effective degree of adaptation D eff (≈0.67) of the average observer in this study.This value is quite low, as based on the very high luminance of the background (760 cd/m 2 ) and Eq. ( 4), one would expect full adaptation.Although adaptation is rarely complete and the median value for the neutral adaptive condition is in agreement with values reported in literature for color constancy under laboratory conditions [5] and for appearance matches (rather than surfaces matches) [52], the results suggest that the viewing conditions were probably still insufficiently representative of real-life situations where higher chromatic adaptation levels are typical.Despite the high luminance and large field of view (50°) of the background scene, as well as the care taken to provide the observer with various cues to the illumination to increase the effective degree of adaptation, it is likely that the observers experienced a mixed adaptation state due to the presence of the dark surround caused by eye and head movements during the course of the experiments.Correcting for the dark surround using the correction factor F in Eq. ( 4), the effective degree in a surround of average luminosity would have been 0.84.Other possible causes for the lower than expected degree of adaptation are that the background scene contained only spectrally unselective, i.e. "gray" objects.It has been shown that color constancy tends to increase as the visual system has several colored surfaces available to estimate the illuminant from [74], for instance through the (near) invariance of cone-excitation ratios under typical illuminant changes [75] or other relational color constancy mechanisms [5,49].However, colored objects were not added to the background scene as to not bias the overall background chromaticity, which might be used to estimate the illuminant [64].Future experiments will therefore involve a more immersive environment with additional cues to the illumination condition.

Summary and conclusions
An asymmetric memory color matching method for deriving corresponding colors was explored.The method has several advantages over other more traditional asymmetric matching methods, the most important being the limited number of matching experiments required to generate an extensive set of corresponding colors, the ease for observers (no training, no switching back and forth between adaptive conditions), the possibility of realistic viewing conditions and the extended set of possible target colors.
In this study a set of corresponding memory colors under 12 'neutral' illumination pairs at a single background luminance of 760 cd/m 2 was obtained.The memory color matching method was found to provide corresponding color data that complies well very with the von Kries law, which states that chromatic adaptation can be thought of as the independent gain regulation of cone related sensors.Prediction errors were comparable to what are considered just-noticeable-differences, supporting the use of memory colors as internal references in the study of chromatic adaptation.
The data was used to test several chromatic adaptation models from literature.A twostep von Kries chromatic adaptation transform with freely optimized degree of adaptation was found to perform the best of all tested CAT models.The worse performance of the other models can be attributed to an overestimation of the effective degree of adaptation of the average observer.
An analysis of the experimental D eff showed that for these neutral illumination conditions D eff is lower than what would be expected from the high luminance background adopted in this study.Although the dark surround adopted is one possible cause, other appearance effects could have contributed as well.Future experiments will involve more immersive environments with additional cues to the illumination condition.
.e. for full adaptation; or in the case where X wB = X 0 .The latter also explains why one-step models have worked quite well in predicting corresponding color data sets published in literature and for many practical applications, as one of the two illumination conditions typically has a chromaticity (often D65 or similar) close to the illuminant independent baseline state.A two-step adaptation model is also the CAT equivalent of what actually happens when corresponding colors under different viewing conditions are calculated using the forward and backward modes of a full color appearance model: one first transforms from condition A to the color attributes under the adopted white point, i.e. the baseline condition, and then performs an inverse transform back to the tristimulus values under condition B.

Fig. 1 .
Fig. 1.Schematic of one-step and two-step chromatic adaptation transforms between corresponding colors under adaptive conditions and A and B. Adaptive condition 0 is baseline condition of Eq. (1).

Fig. 2 .
Fig. 2. (a) Illustration of the background scene under 4000 K illumination with the cube in a magenta starting chromaticity (see experimental procedure).(b) Side view schematic of the experimental setup.