Abstract
A multi-cue illumination estimation method based on tree-structured group joint sparse representation is proposed. Tests show that the proposed method works better than existing methods, most of which are based on using only a single cue type, for example, a binarized color histogram or simple image statistic such as the mean RGB. Most existing illumination estimation methods make their estimates using only one of three kinds of cues. They differ in which cue type they use, but the chosen cue is either based on (1) properties of the low-level RGB color distribution, (2) mid-level initial illuminant estimates provided by subordinate methods, or (3) high-level knowledge of scene content (e.g., indoor versus outdoor scene). The proposed multi-cue method combines the information provided by cues of all three of these types within the framework of a tree-structured group joint sparse representation (TGJSR). In TGJSR, the training data is grouped into a tree of subgroups. A test image under an unknown illuminant has its features reconstructed in terms of a joint sparse representation model derived from the grouped training data. The test image’s illumination is then estimated based on the weights involved in the joint sparse representation model. As a general framework, the proposed TGJSR framework can also easily be extended to incorporate any new features or cues that might be discovered in the future for illumination estimation.
Similar content being viewed by others
Notes
\(O_1 =\frac{R-G}{\sqrt{2}},O_2 =\frac{R+G-2B}{\sqrt{6}},O_3 =\frac{R+B+G}{\sqrt{3}}\)
References
Barnard, K., Cardei, V., & Funt, B. (2002). A comparison of computational color constancy algorithms-part 1: Methodology and experiments with synthesized data. IEEE Transactions on Image Processing, 11(9), 972–983.
Barnard, K., Martin, L., Coath, A., & Funt, B. (2002). Comparison of computational color constancy algorithms-part 2: Experiments with Image Data. IEEE Transactions on Image Processing, 11(9), 985–996.
Beigpour, S., Riess, C., Weijer, J. V., & Angelopoulou, E. (2014). Multi-Illuminant estimation with conditional random fields. IEEE Transactions on Image Processing, 23(1), 83–96.
Bengio, S., Pereira, F., Singer, Y., Strelow, D. (2009). Group sparse coding. In: Proceedings of neural information processing systems conference (pp. 1–8).
Bianco, S., Gasparini, F., & Schettini, R. (2008). Consensus-based framework for illuminant chromaticity estimation. Journal of Electronic Imaging, 17(2), 023013.
Bianco, S., Ciocca, G., Cusano, C., & Schettini, R. (2009). Improving Color Constancy using Indoor-outdoor Image Classification. IEEE Transactions on Image Processing, 17(12), 2381–2392.
Bianco, S., Ciocca, G., Cusano, C., & Schettini, R. (2009). Automatic color constancy algorithm selection and combination. Pattern Recognition, 43(3), 695–705.
Brainard, D. H., & Freeman, W. T. (1997). Bayesian color constancy. Journal of the Optical Society of America A, 14(7), 1393–1411.
Buchsbaum, G. (1980). A spatial processor model for object colour perception. Journal of the Franklin Institute, 310(1), 1–26.
Cardei, V., & Funt, B. (1999). Committee-based color constancy. In: Proceedings of IS&T/SID color imaging conference (CIC) (pp. 311–313).
Cardei, V., Funt, B., & Barnard, K. (2002). Estimating the scene illumination chromaticity using a neural network. Journal of the Optical Society of America A, 19(12), 2374–2386.
Chakrabarti, A., Hirakawa, K., & Zickler, T. (2012). Color constancy with spatio-spectral statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(8), 1509–1519.
Chen, X., Pan, W., Kwok, J., & Garbonell, J. (2009). Accelerated gradient method for multi-task sparse learning problem. In: Proceedings of international conference of data mining (pp. 746–751).
Ciurea, F., & Funt, B. (2003). A large image database for color constancy research. In: Proceedings of IS&T/SID color imaging conference (pp. 160–164).
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
Finlayson, G.G. (2013). Corrected-moment illuminant estimation. In: Proceedings of international conference on computer vision (ICCV) (pp. 1904–1911).
Finlayson, G., & Trezzi, E. (2004). Shades of gray and color constancy. In Proceedings of IS&T/SID color imaging conference (pp. 37–41).
Finlayson, G., Hordley, S., & Hubel, P. (2001). Color by correlation: A simple, unifying framework for color constancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), 1209–1221.
Forsyth, D. A. (1990). A novel algorithm for color constancy. International Journal of Computer Vision, 5(1), 5–36.
Foster, D. H. (2010). Colour constancy. Vision Research, 51, 674–700.
Funt, B., & Mosny, M. (2012). Removing outliers in illumination estimation. In: Proceedings of IS&T color imaging conference (pp. 105–110).
Funt, B., & Shi, L. (2012). MaxRGB reconsidered. Journal of Imaging Science and Technology, 56(2), 020501-1–020501-10.
Gao, S., Chia, L., & Tsang, I. (2011). Multi-layer group sparse coding for concurrent image classification and annotation. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 2809–2816).
Gao, S., Yang, K., Li, C., & Li, Y. (2013). A color constancy model with double-opponency mechanisms. In: Proceedings of international conference on computer vision (pp. 929–936).
Gehler, P. V., Rother, C., Blake, A., & Minka, T. (2008). Bayesian color constancy revisited. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1–8).
Geusebroek, J. M., & Smeulders, A. W. M. (2005). A six stimulus theory for stochastic texture. International Journal of Computer Vision, 62(1–2), 7–16.
Gijsenij, A., Gevers, T., & Lucassen, M. (2009). A Perceptual analysis of distance measures for color constancy algorithms. Journal of the Optical Society of America A, 26(10), 2243–2256.
Gijsenij, A., Gevers, T., & van de Weijer, J. (2010). Generalized gamut mapping using image derivative structures for color constancy. International Journal of Computer Vision, 86(2–3), 127–139.
Gijsenij, A., Gevers, T., & van de Weijer, J. (2011). Computational color constancy: Survey and experiments. IEEE Transactions on Image Processing, 20(9), 2475–2489.
Gijsenij, A., Gevers, T., & van de Weijer, J. (2011). Improving color constancy by photometric edge weighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(5), 918–929.
Gijsenij, A., Lu, R., & Gevers, T. (2012). Color constancy for multiple light sources. IEEE Transactions on Image Processing, 21(2), 697–707.
Gijsenij, A., & Gevers, T. (2011). Color constancy using natural image statistics and scene semantics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4), 687–698.
Jenatton, R., Audibert, J., & Bach, F. (2011). Structured variable selection with sparsity-inducing norms. Journal of Machine Learning Research, 12(10), 2777–2824.
Joze, H. R. V., & Drew, M. S. (2014). Exemplar-based colour constancy and multiple illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(5), 860–873.
Land, E. H. (1977). The retinex theory of color vision. Scientific America, 237(6), 108–128.
Li, B., Xiong, W., Hu, W., & Peng, H. (2013). Illumination estimation based on bilayer sparse coding. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1423–1429).
Li, B., Xiong, W., & Xu, D. (2010). A supervised combination strategy for illumination chromaticity estimation. ACM Transactions on Applied Perception, 8(1), 5.
Li, B., Xiong, W., Hu, W., & Funt, B. (2014). Evaluating combinational illumination estimation methods on real-world images. IEEE Transactions on Image Processing, 23(3), 1194–1209.
Liu, J., & Ye, J. (2010). Moreau-Yosida regularization for grouped tree structure learning. In: Proceedings of neural information processing systems conference (pp. 1–8).
Logvinenko, A. D., Funt, B., & Godau, C. (2014). Metamer mismatching. IEEE Transactions on Image Processing, 23(1), 34–43.
Lowe, D. G. (2004). Distinctive image features from scale-invariant key-points. International Journal of Computer Vision, 60(2), 91–110.
Lu, R., Gijsenij, A., Gevers, T., Nedovic, V., & Xu, D. (2009). Color Constancy using 3D Scene Geometry. In: Proceedings of international conference on computer vision (pp. 1749-1756).
Nedovic, V., Smeulders, A. W. M., Redert, A., & Geusebroek, J. M. (2010). Stages as models of scene geometry. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), 1673–1687.
Nesterov, Y. (2007). Gradient methods for minimizing composite objective function. CORE, Catholic University Louvain, Louvain-la-Neuve, Technical Report.
Parraga, C. A., Vazquez-Corral, J., & Vanrell, M. (2009). A new cone activation-based natural images dataset. Perception, 36, 180.
Ruderman, D. L., Cronin, T. W., & Chiao, C. C. (1998). Statistics of cone responses to natural images: implications for visual coding. Journal of the Optical Society of America A, 15(8), 2036–2045.
Shi, L. (2012). Personal communication.
Shi, L., & Funt, B. (2011). Re-processed version of the gehler color constancy dataset of 568 Images. Retrieved from http://www.cs.sfu.ca/~colour/data/
Tan, R., Nishino, K., & Ikeuchi, K. (2004). Color constancy through inverse-intensity chromaticity space. Journal of the Optical Society of America A, 21(3), 321–334.
Tseng, P. (2008). On accelerated proximal gradient methods for convex-concave optimization. SIAM Journal on Optimization, 6, 1–20.
van de Weijer, J., Schmid, C., & Verbeek, J. (2007). Using High-level visual information for color constancy. In: Proceedings of international conference on computer vision (pp. 1–8).
van de Weijer, J., Gevers, T., & Gijsenij, A. (2007). Edge based color constancy. IEEE Transactions on Image Processing, 16(9), 2207–2214.
Vazquez-Corral, J., Alejandro Párraga, C., Vanrell, M., & Baldrich, R. (2009). Color constancy algorithms: Psychophysical evaluation on a new dataset. Journal of Imaging Science and Technology, 53(3), 031105–031109.
Vazquez-Corral, J., Vanrell, M., Baldrich, R., & Tous, F. (2012). Color constancy by category correlation. IEEE Transactions on Image Processing, 21(4), 1997–2007.
Wright, J., Ma, Y., Mairal, J., Sapiro, G. ( 2010). Sparse representation for computer vision and pattern recognition. In: Proceedings of the IEEE.
Xiong, W., & Funt, B. (2006). Estimating illumination chromaticity via support vector regression. Journal of Imaging Science and Technology, 50(4), 341–348.
Yang, A., Zhou, Z., Balasubramanian, A., Sastry, S., & Ma, Y. (2013). Fast L1-minimization algorithms for robust face recognition. IEEE Transactions on Image Processing, 22(8), 3234–3246.
Yosida, K. (1964). Functional analysis. Berlin: Springer.
Yuan, X., & Yan, S. (2010). Visual classification with multi-task joint sparse representation. In: Proceedings of IEEE conference on computer vision and pattern recognition (pp. 3493–3500).
Acknowledgments
This work was supported by the National Nature Science Foundation of China (Nos. 61370038, and 61472421), the 973 Basic Research Program of China (No. 2014CB349303), and Chinese National Programs for High Technology Research and Development (863 Program) (Nos. 2012AA012503 and 2012AA012504), as well as the Natural Sciences and Engineering Research Council of Canada.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Ko Nishino.
Rights and permissions
About this article
Cite this article
Li, B., Xiong, W., Hu, W. et al. Multi-Cue Illumination Estimation via a Tree-Structured Group Joint Sparse Representation. Int J Comput Vis 117, 21–47 (2016). https://doi.org/10.1007/s11263-015-0844-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-015-0844-7