Abstract
Specular highlight widely exists in daily life. Its strong brightness influences the recognition of text and graphic patterns in images, especially for documents and cards. In this paper, we propose a coarse-to-fine dynamic association learning method for specular highlight detection and removal. Specifically, based on the dichromatic reflection model, we first use a sub-network to separate the specular highlight layer and locate the regions of the highlight. Instead of directly subtracting the estimated specular highlight component from the raw image to get the highlight removal result, we design an associated learning module (ALM) together with a second-stage sub-network to restore the color distortion of the specular highlight layer removal. Our ALM respectively extracts features from the specular highlight part and non-specular highlight part to improve the color restoration. We conducted extensive evaluation experiments and the ablation study on the synthetic dataset and the real-world dataset. Our method achieved 36.09 PSNR and 97% SSIM on SHIQ dataset, along with 28.90 PSNR and 94% SSIM on SD1 dataset, which outperformed the SOTA methods.
J. Shen and H. Guan—Contribute equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akashi, Y., Okatani, T.: Separation of reflection components by sparse non-negative matrix factorization. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9007, pp. 611–625. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16814-2_40
Ali, S.G., et al.: Experimental protocol designed to employ nd: YAG laser surgery for anterior chamber glaucoma detection via UBM. IET Image Proc. 16(8), 2171–2179 (2022)
Fu, G., Zhang, Q., Lin, Q., Zhu, L., Xiao, C.: Learning to detect specular highlights from real-world images. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 1873–1881 (2020)
Fu, G., Zhang, Q., Zhu, L., Li, P., Xiao, C.: A multi-task network for joint specular highlight detection and removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7752–7761 (2021)
Fu, G., Zhang, Q., Zhu, L., Lin, Q., Wang, Y., Fan, S., Xiao, C.: Towards high-resolution specular highlight detection. Int. J. Comput. Vis. 1–23 (2023)
Funke, I., Bodenstedt, S., Riediger, C., Weitz, J., Speidel, S.: Generative adversarial networks for specular highlight removal in endoscopic images. In: Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 10576, pp. 8–16. SPIE (2018)
He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)
Hou, S., Wang, C., Quan, W., Jiang, J., Yan, D.-M.: Text-aware single image specular highlight removal. In: Ma, H., et al. (eds.) PRCV 2021. LNCS, vol. 13022, pp. 115–127. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88013-2_10
Hu, G., Zheng, Y., Yan, H., Hua, G., Yan, Y.: Mask-guided cycle-GAN for specular highlight removal. Pattern Recogn. Lett. 161, 108–114 (2022)
Hu, X., Zhu, L., Fu, C.W., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7454–7462 (2018)
Huang, Z., Hu, K., Wang, X.: M2-Net: multi-stages specular highlight detection and removal in multi-scenes. arXiv preprint arXiv:2207.09965 (2022)
Ikeuchi, K., Miyazaki, D., Tan, R.T., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. In: Digitally Archiving Cultural Objects, pp. 353–384. Springer, Boston, MA (2008). https://doi.org/10.1007/978-0-387-75807_17
Jiang, K., et al.: Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8346–8355 (2020)
Li, R., Pan, J., Si, Y., Yan, B., Hu, Y., Qin, H.: Specular reflections removal for endoscopic image sequences with adaptive-RPCA decomposition. IEEE Trans. Med. Imaging 39(2), 328–340 (2019)
Lin, J., El Amine Seddik, M., Tamaazousti, M., Tamaazousti, Y., Bartoli, A.: Deep multi-class adversarial specularity removal. In: Felsberg, M., Forssén, P.-E., Sintorn, I.-M., Unger, J. (eds.) SCIA 2019. LNCS, vol. 11482, pp. 3–15. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20205-7_1
Liu, Y., Yuan, Z., Zheng, N., Wu, Y.: Saturation-preserving specular reflection separation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3725–3733 (2015)
Madessa, A.H., Dong, J., Gan, Y., Gao, F.: A deep learning approach for specular highlight removal from transmissive materials. Expert. Syst. 40(2), e12598 (2023)
Meka, A., Maximov, M., Zollhoefer, M., Chatterjee, A., Theobalt, C.: Live Intrinsic Material Estimation. IEEE (2018)
Muhammad, S., Dailey, M.N., Farooq, M., Majeed, M.F., Ekpanyapong, M.: Spec-Net and spec-CGAN: Deep learning models for specularity removal from faces. Image Vis. Comput. 93, 103823 (2020)
Phong, B.T.: Illumination for computer generated pictures. Commun. ACM 18(6), 311–317 (1975)
Shen, H.L., Zheng, Z.H.: Real-time highlight removal using intensity ratio. Appl. Opt. 52(19), 4483–4493 (2013)
Tominaga, S.: Dichromatic reflection models for a variety of materials. Color Res. Appl. 19(4), 277–285 (1994)
Wang, X., Tao, C., Tao, X., Zheng, Z.: SIHRNET: a fully convolutional network for single image highlight removal with a real-world dataset. J. Electron. Imaging 31(3), 033013–033013 (2022)
Xu, J., Liu, S., Chen, G., Liu, Q.: Highlight detection and removal method based on bifurcated-CNN. In: Liu, H., et al. Intelligent Robotics and Applications. ICIRA 2022. Lecture Notes in Computer Science, vol. 13458, pp. 307–318. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-13841-6_29
Yamamoto, T., Nakazawa, A.: General improvement method of specular component separation using high-emphasis filter and similarity function. ITE Trans. Media Technol. Appl. 7(2), 92–102 (2019)
Yang, Q., Tang, J., Ahuja, N.: Efficient and robust specular highlight removal. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1304–1311 (2014)
Zhang, W., Zhao, X., Morvan, J.M., Chen, L.: Improving shadow suppression for illumination robust face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41(3), 611–624 (2018)
Acknowledgement
This research is supported by the National Natural Science Foundation of China (No. 62172309).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shen, J., Guan, H., Tao, S., Yan, K., Zhou, F., Luo, F. (2024). Specular Highlight Detection and Removal Based on Dynamic Association Learning. In: Sheng, B., Bi, L., Kim, J., Magnenat-Thalmann, N., Thalmann, D. (eds) Advances in Computer Graphics. CGI 2023. Lecture Notes in Computer Science, vol 14495. Springer, Cham. https://doi.org/10.1007/978-3-031-50069-5_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-50069-5_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-50068-8
Online ISBN: 978-3-031-50069-5
eBook Packages: Computer ScienceComputer Science (R0)