Abstract
Understanding blur from a single defocused image contains two tasks of defocus detection and deblurring. This paper makes the earliest effort to jointly learn both defocus detection and deblurring without using pixel-level defocus detection annotation and paired defocus deblurring ground truth. We build on the observation that these two tasks are supplementary to each other: Defocus detection can segment the focused area from the defocused image to guide the defocus deblurring; Conversely, to achieve better defocus deblurring, an accurate defocus detection as the guide is essential. Therefore, we implement an adversarial promoting learning framework to jointly handle defocus detection and defocus deblurring. Specifically, a defocus detection generator \(G_{ws}\) is implemented to represent the defocused image as a layered composition of two elements: defocused image \(I_{df}\) and a focused image \(I_f\). Then, \(I_{df}\) and \(I_f\) are fed into a self-referenced defocus deblurring generator \(G_{sr}\) to generate a deblurred image. Two generators of \(G_{ws}\) and \(G_{sr}\) are optimized alternately in an adversarial manner against a discriminator D with unpaired realistic fully-clear images. Thus, \(G_{sr}\) will produce a deblurred image to fool D, and \(G_{ws}\) is forced to generate an accurate defocus detection map to effectively guide \(G_{sr}\). Comprehensive experiments on two defocus detection datasets and one defocus deblurring dataset demonstrate the effectiveness of our framework. Code and model are available at: https://github.com/wdzhao123/APL.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abuolaim, A., Brown, M.S.: Defocus deblurring using dual-pixel data. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12355, pp. 111–126. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58607-2_7
Alireza Golestaneh, S., Karam, L.J.: Spatially-varying blur detection based on multiscale fused and sorted transform coefficients of gradient magnitudes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5800–5809 (2017)
Bulat, A., Yang, J., Tzimiropoulos, G.: To learn image super-resolution, use a GAN to learn how to do image degradation first. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 185–200 (2018)
Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3155–3164 (2018)
Cun, X., Pun, C.-M.: Defocus blur detection via depth distillation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12358, pp. 747–763. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58601-0_44
Ding, J., Huang, Y., Liu, W., Huang, K.: Severely blurred object tracking by learning deep image representations. IEEE Trans. Circuits Syst. Video Technol. 26(2), 319–331 (2015)
Guo, Q., Feng, W., Gao, R., Liu, Y., Wang, S.: Exploring the effects of blur and deblurring to visual object tracking. IEEE Trans. Image Process. 30, 1812–1824 (2021)
Gur, S., Wolf, L.: Single image depth estimation trained via depth from defocus cues. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 7683–7692 (2019)
Jiang, P., Ling, H., Yu, J., Peng, J.: Salient region detection by UFO: uniqueness, focusness and objectness. In: IEEE International Conference on Computer Vision, pp. 1976–1983 (2013)
Karaali, A., Jung, C.R.: Edge-based defocus blur estimation with adaptive scale selection. IEEE Trans. Image Process. 27(3), 1126–1137 (2017)
Kim, B., Son, H., Park, S.J., Cho, S., Lee, S.: Defocus and motion blur detection with deep contextual features. In: Computer Graphics Forum, vol. 37, pp. 277–288 (2018)
Kupyn, O., Martyniuk, T., Wu, J., Wang, Z.: DeblurGAN-v2: deblurring (orders-of-magnitude) faster and better. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8878–8887 (2019)
Lee, J., Lee, S., Cho, S., Lee, S.: Deep defocus map estimation using domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12222–12230 (2019)
Lee, J., Son, H., Rim, J., Cho, S., Lee, S.: Iterative filter adaptive network for single image defocus deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2034–2042 (2021)
Li, J., et al.: Layer-output guided complementary attention learning for image defocus blur detection. IEEE Trans. Image Process. 30, 3748–3763 (2021)
Luo, B., Cheng, Z., Xu, L., Zhang, G., Li, H.: Blind image deblurring via superpixel segmentation prior. IEEE Trans. Circuits Syst. Video Technol. 32, 1467–1482 (2021)
Pan, L., Dai, Y., Liu, M., Porikli, F., Pan, Q.: Joint stereo video deblurring, scene flow estimation and moving object segmentation. IEEE Trans. Image Process. 29, 1748–1761 (2019)
Pang, Y., Zhu, H., Li, X., Li, X.: Classifying discriminative features for blur detection. IEEE Trans. Cybern. 46(10), 2220–2227 (2015)
Park, J., Tai, Y.W., Cho, D., So Kweon, I.: A unified approach of multi-scale deep and hand-crafted features for defocus estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1736–1745 (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sakurikar, P., Narayanan, P.: Composite focus measure for high quality depth maps. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1614–1622 (2017)
Shi, J., Xu, L., Jia, J.: Discriminative blur detection features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2965–2972 (2014)
Shi, J., Xu, L., Jia, J.: Just noticeable defocus blur detection and estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 657–665 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, pp. 1–14 (2014)
Son, H., Lee, J., Cho, S., Lee, S.: Single image defocus deblurring using kernel-sharing parallel atrous convolutions. In: Proceedings of the IEEE International Conference on Computer Vision (2021)
Su, B., Lu, S., Tan, C.L.: Blurred image region detection and classification. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 1397–1400 (2011)
Tai, Y.W., Brown, M.S.: Single image defocus map estimation using local contrast prior. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 1797–1800. IEEE (2009)
Tang, C., Liu, X., An, S., Wang, P.: BR\(^2\)Net: defocus blur detection via bidirectional channel attention residual refining network. IEEE Trans. Multimed. 23, 624–635 (2020)
Tang, C., et al.: DeFusionNET: defocus blur detection via recurrently fusing and refining discriminative multi-scale deep features. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2020)
Tang, C., et al.: R\(^2\)MRF: defocus blur detection via recurrently refining multi-scale residual features. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12063–12070 (2020)
Tang, C., Wu, J., Hou, Y., Wang, P., Li, W.: A spectral and spatial approach of coarse-to-fine blurred image region detection. IEEE Signal Process. Lett. 23(11), 1652–1656 (2016)
Tang, C., Zhu, X., Liu, X., Wang, L., Zomaya, A.: DeFusionNet: defocus blur detection via recurrently fusing and refining multi-scale deep features. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2700–2709 (2019)
Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8174–8182 (2018)
Wang, J., Li, X., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1788–1797 (2018)
Wang, X., Yu, K., Dong, C., Loy, C.C.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 606–615 (2018)
Xiong, W., et al.: Foreground-aware image inpainting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5840–5848 (2019)
Yi, X., Eramian, M.: LBP-based segmentation of defocus blur. IEEE Trans. Image Process. 25(4), 1626–1638 (2016)
Zhang, K., Zuo, W., Zhang, L.: Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3271 (2018)
Zhang, N., Yan, J.: Rethinking the defocus blur detection problem and a real-time deep DBD model. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12355, pp. 617–632. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58607-2_36
Zhang, S., Shen, X., Lin, Z., Mech, R., Costeira, J.P., Moura, J.M.F.: Learning to understand image blur. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6586–6595 (2018)
Zhang, X., Wang, R., Jiang, X., Wang, W., Gao, W.: Spatially variant defocus blur map estimation and deblurring from a single image. J. Vis. Commun. Image Represent. 35, 257–264 (2016)
Zhang, Z., Liu, Y., Xiong, Z., Li, J., Zhang, M.: Focus and blurriness measure using reorganized DCT coefficients for an autofocus application. IEEE Trans. Circuits Syst. Video Technol. 28(1), 15–30 (2016)
Zhao, F., Lu, H., Zhao, W., Yao, L.: Image-scale-symmetric cooperative network for defocus blur detection. IEEE Trans. Circuits Syst. Video Technol. 32(5), 2719–2731 (2021)
Zhao, W., Hou, X., He, Y., Lu, H.: Defocus blur detection via boosting diversity of deep ensemble networks. IEEE Trans. Image Process. 30, 5426–5438 (2021)
Zhao, W., Hou, X., Yu, X., He, Y., Lu, H.: Towards weakly-supervised focus region detection via recurrent constraint network. IEEE Trans. Image Process. 29, 1356–1367 (2019)
Zhao, W., Shang, C., Lu, H.: Self-generated defocus blur detection via dual adversarial discriminators. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6933–6942 (2021)
Zhao, W., Zhao, F., Wang, D., Lu, H.: Defocus blur detection via multi-stream bottom-top-bottom fully convolutional network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3080–3088 (2018)
Zhao, W., Zhao, F., Wang, D., Lu, H.: Defocus blur detection via multi-stream bottom-top-bottom network. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 1884–1897 (2020)
Zhao, W., Zheng, B., Lin, Q., Lu, H.: Enhancing diversity of defocus blur detectors via cross-ensemble network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8905–8913 (2019)
Acknowledgements
This work is supported by National Natural Science Foundation of China under Grant Nos. 62176038 and U1903215, and Science and Technology Star of Dalian under Grant No. 2021RQ054.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, W., Wei, F., He, Y., Lu, H. (2022). United Defocus Blur Detection and Deblurring via Adversarial Promoting Learning. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13690. Springer, Cham. https://doi.org/10.1007/978-3-031-20056-4_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-20056-4_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20055-7
Online ISBN: 978-3-031-20056-4
eBook Packages: Computer ScienceComputer Science (R0)