Skip to main content
SpringerLink
Log in

A multi-level feature weight fusion model for salient object detection

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Although the Fully Convolutional Neural Networks (FCNs) has achieved good performance in salient object detection, there are problems, such as fuzzy boundary and unsatisfactory performance in complex scenes. Hence, how to better integrate multi-level convolution feature requires further investigation. This paper proposes a salient object detection algorithm, which uses Gram matrix and its F norm to weigh the importance of each multi-level feature map and uses weight to fuse multi-level prediction results recursively, finally generate the final saliency map. The algorithm evaluates the importance of different depth multi-level feature maps by calculating the Gram matrix's F norm of feature tensor slices. The multi-level feature maps are fused effectively according to the weight. It reduces the loss of multi-level prediction results during fusion, and preserves the spatial details. Besides, to achieve a more accurate boundary, a deep supervision is used to optimize salient feature maps’ results. Pixel-level supervision information from ground truth will guide each layer’s prediction. Experiments on five benchmark data sets demonstrate that the proposed method performs well in various scenes, especially in complex scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The raw/processed data required to reproduce these findings cannot be shared at this time as the data also forms part of an ongoing study.

References

  1. Chang, C.: Research on natural scene image classification algorithm based on saliency detection [D], pp. 1–55. Wuhan University of Technology, Wuhan (2018)

    Google Scholar 

  2. Muratov, O., Zontone, P., Boato, G., et al.: A segment-based image saliency detection [C]. IEEE International Conference on Acoustics. IEEE, pp.1217–1220 (2011)

  3. Runchun, Ye.: The optimization model of saliency detection and its application in image compression [D], pp. 1–45. University of Science and Technology of China, Hefei (2018)

    Google Scholar 

  4. Cox, I.J., Kilian, J., Leighton, F.T., et al.: Secure spread spectrum watermarking for multimedia [J]. IEEE Trans. Image Process. 6(12), 1673–1687 (1997)

    Article  Google Scholar 

  5. Achanta, R., Estrada, F., Wils, P., et al.: Salient Region Detection and Segmentation [C]. International Conference on Computer Vision Systems, pp. 66–75. Springer, Berlin (2008)

    Google Scholar 

  6. Ma, Y.-F., Zhang, H.-J.: Contrast-based image attention analysis by using fuzzy growing [C]. Proceedings of the Eleventh ACM International Conference on Multimedia, pp. 374–381 (2003)

  7. Liu, T., Yuan, Z., Sun, J., et al.: Learning to detect a salient object [J]. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 353–367 (2010)

    Google Scholar 

  8. Wei, Y., Wen, F., Zhu, W., et al.: Geodesic saliency using background priors [C]. European Conference on Computer Vision, pp. 29–42. Springer, Berlin (2012)

    Google Scholar 

  9. Tong, N., Lu, H., Ruan, X., et al.: Salient object detection via bootstrap learning [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1884–1892 (2015)

  10. Cheng, M.M., Mitra, N.J., Huang, X., et al.: Global contrast based salient region detection [J]. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2014)

    Article  Google Scholar 

  11. Wang L, Lu H, Ruan X, et al. Deep networks for saliency detection via local estimation and global search [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3183–3192 (2015)

  12. Lee, G., Tai, Y.W., Kim, J.: Deep saliency with encoded low level distance map and high level features [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 660–668 (2016)

  13. Li, H., Chen, J., Lu, H., et al.: CNN for saliency detection with low-level feature integration [J]. Neurocomputing 226, 212–220 (2017)

    Article  Google Scholar 

  14. Wang, T., Borji, A., Zhang, L., et al.: A stagewise refinement model for detecting salient objects in images [C]. Proceedings of the IEEE International Conference on Computer Vision, pp. 4019–4028 (2017)

  15. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

  16. Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition [J]. arXiv preprint arXiv:1409.1556, (2014)

  17. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  18. Zhang, P., Wang, D., Lu, H., et al.: Amulet: Aggregating multi-level convolutional features for salient object detection [C]. Proceedings of the IEEE International Conference on Computer Vision. 2017: 202–211.

  19. Hariharan, B., Arbeláez, P., Girshick, R., et al.: Hypercolumns for object segmentation and fine-grained localization [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 447–456 (2015)

  20. Huang, J., Rathod, V., Sun, C., et al.: Speed/accuracy trade-offs for modern convolutional object detectors [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7310–7311 (2017)

  21. Liu, N., Han, J.W.: Dhsnet: deep hierarchical saliency network for salient object detection [C]. IEEE Conference on Computer Vision and Pattern Recognition, pp. 678–686. IEEE Computer Society Press, Los Alamitos (2016)

    Google Scholar 

  22. Zhang, L., Dai, J., Lu, H., et al.: A bi-directional message passing model for salient object detection [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1741–1750 (2018)

  23. Zhao, H., Shi, J., Qi, X., et al.: Pyramid scene parsing network [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)

  24. Hu, X., Zhu, L., Qin, J., et al.: Recurrently aggregating deep features for salient object detection [C]. Proceedings of the AAAI Conference on Artificial Intelligence, pp. 32–39 (2018)

  25. Gatys, L., Ecker, A., Bethge, M.: A neural algorithm of artistic style [J]. J. Vis. 16(12), 326–326 (2016)

    Article  Google Scholar 

  26. Wang, L., Lu, H., Wang, Y., et al.: Learning to detect salient objects with image-level supervision [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 136–145 (2017).

  27. Li, Y., Hou, X., Koch, C., et al.: The secrets of salient object segmentation [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287 (2014)

  28. Movahedi, V., Elder, J.H.: Design and perceptual validation of performance measures for salient object segmentation [C]. Computer Vision & Pattern Recognition Workshops. IEEE, pp. 49–56 (2010)

  29. Li, G., Yu, Y.: Visual saliency based on multiscale deep features [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)

  30. Yang, C., Zhang, L., Lu, H., et al.: Saliency detection via graph-based manifold ranking [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)

  31. Lecun, Y., Bottou, L.: Gradient-based learning applied to document recognition [J]. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  32. Borji, A., Cheng, M.M., Jiang, H., et al.: Salient object detection: a benchmark [J]. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Funding

This research was supported by National Natural Science Foundation of China (Grant No. 62172132).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, 310018, China

    Zhang Shanqing, Chen Yujie, Meng Yiheng, Lu Jianfeng, Li Li & Bai Rui

  2. Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou, 310018, China

    Bai Rui

Authors
  1. Zhang Shanqing
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Yujie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meng Yiheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lu Jianfeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Li Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bai Rui
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors reviewed the manuscript.

Corresponding author

Correspondence to Zhang Shanqing.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shanqing, Z., Yujie, C., Yiheng, M. et al. A multi-level feature weight fusion model for salient object detection. Multimedia Systems 29, 887–895 (2023). https://doi.org/10.1007/s00530-022-01018-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-01018-1

Keywords

Search

Navigation