双特征流融合和边界感知的显著性目标检测

doi:10.3778/j.issn.1002-8331.2301-0234

摘要/Abstract

摘要： 显著性目标检测是计算机视觉领域的热门研究方向之一，许多基于深度学习的检测算法虽然已经取得了显著的成果，但是仍然存在待测目标漏检误检和边界模糊等问题。针对这些问题提出了一种基于双特征流融合和边界感知的目标检测算法，通过改变输入图像尺寸来丰富多尺度信息，并自顶向下逐层聚合特征得到精细的预测结果。首先将输入图像调整为两种不同分辨率分别送入编码器，提取丰富的多层级特征形成双特征流；其次将双特征流自顶向下逐层融合，生成由粗到细的显著图；最后构建了边界感知结构，凭借上下文语义信息的指导生成精细的物体轮廓。在五个公开数据集上进行了大量实验，实验结果表明，所提算法在结构相似性[(Sm)]等多个指标上取得了更高的检测精度，生成的显著图目标完整且边缘清晰。

关键词: 显著性目标检测, 全卷积神经网络, 多尺度学习, 双特征流融合, 边界感知

Abstract: Salient object detection is a popular research area in computer vision. Numerous deep learning-based methods have been proposed, and have shown great potential. However, there are some problems with the detection, such as false detection or fuzzy edges. To address these problems, this paper proposes a novel dual-stream fusion and edge-aware network for salient object detection. By utilizing the multi-scale information and features aggregating, the model can obtain the fine-grained results. Firstly, the input image is modified into two different scales to feed to the encoder of the network separately. In this way, the model can extract the abundant multi-level features from the dual-feature stream. Secondly, in the stage of the decoder, the dual-stream features are gradually integrated from top to bottom, to generate the coarse-to-fine salient map. Moreover, a dual-path edge-aware structure is designed, which can generate the subtle boundary by fusing the semantic contextual information. Finally, the edge refinement module is used to improve the salient object boundaries, and obtain the final results. Extensive experiments are conducted on five public datasets, the results show that the proposed method achieves higher accuracy in terms of structural similarity (Sm), and generates high quality salient maps with more complete objects and distinct edges.

Key words: salient object detection, full convolutional neural network, multi-scale learning, dual-stream fusion, edge-aware

杨鑫, 朱恒亮, 毛国君. 双特征流融合和边界感知的显著性目标检测[J]. 计算机工程与应用, 2024, 60(10): 227-236.

YANG Xin, ZHU Hengliang, MAO Guojun. Dual-Stream Fusion and Edge-Aware Network for Salient Object Detection[J]. Computer Engineering and Applications, 2024, 60(10): 227-236.

参考文献

[1] FAN D P, CHENG M M, LIU J J, et al. Salient objects in clutter: bringing salient object detection to the foreground [C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 186-202.
[2] MARGOLIN R, ZELNIK-MANOR L, TAL A. How to evaluate foreground maps? [C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014: 248-255.
[3] ZHANG P, ZHUO T, HUANG W, et al. Online object tracking based on CNN with spatial-temporal saliency guided sampling[J]. Neurocomputing, 2017, 257(8): 115-127.
[4] ABDULMUNEM A, LAI Y K, SUN X F. Saliency guided local and global descriptors for effective action recognition[J]. Computational Visual Media, 2016, 2(1): 97-106.
[5] HOYER L, MUNOZ M, KATIYAR P, et al. Grid saliency for context explanations of semantic segmentation[C]//Advances in Neural Information Processing Systems 32, 2019: 6459-6470.
[6] 徐光达, 毛国君. 多层级特征融合的无人机航拍图像目标检测[J]. 计算机科学与探索, 2023, 17(3): 635-645.
XU G D, MAO G J. Aerial image object detection of UAV based on multi-level feature fusion[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(3): 635-645.
[7] 李斌, 李亚霖, 朱新山, 等. 基于注意力机制与特征平衡的变电站多目标检测[J]. 电网技术, 2022, 46(6): 2122-2132.
LI B, LI Y L, ZHU X S, et al. Multi-target detection in substation scence based on attention mechanism and feature balance[J]. Power System Technology, 2022, 46(6): 2122-2132.
[8] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3431-3440.
[9] 王剑哲, 吴秦. 坐标注意力特征金字塔的显著性目标检测算法[J]. 计算机科学与探索, 2023, 17(1): 154-165.
WANG J Z, WU Q. Salient object detection based on coordinate attention feature pyramid[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(1): 154-165.
[10] 罗会兰, 袁璞, 童康. 基于深度学习的显著性目标检测方法综述[J]. 电子学报, 2021, 49(7): 1417-1427.
LUO H L, YUAN P, TONG K. Review of the methods for salient object detection based on deep learning[J]. Chinese Journal of Electronics, 2021, 49(7): 1417-1427.
[11] MA M C, XIA C Q, LI J. Pyramidal feature shrinking for salient object detection[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, 2021: 2311-2318.
[12] ZHUGE M C, FAN D P, LIU N, et al. Salient object detection via integrity learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(3): 3738-3752.
[13] ZHAO Z R, XIA C Q, XIE C X, et al. Complementary trilateral decoder for fast and accurate salient object detection[C]//Proceedings of the 29th ACM International Conference on Multimedia, 2021: 4967-4975.
[14] DING X H, GUO Y C, DING G G, et al. ACNet: strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, 2019: 1911-1920.
[15] CHEN G, LIU S J, SUN Y J, et al. Camouflaged object detection via context-aware cross-level fusion[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(10): 6981-6993.
[16] PANG Y W, ZHAO X Q, ZHANG L H, et al. Multi-scale interactive network for salient object detection[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 9413-9422.
[17] ZHAO Y F, LI J, ZHANG Y, et al. Multi-class part parsing with joint boundary-semantic awareness[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, 2019: 9177-9186.
[18] LI X, YANG F, CHENG H, et al. Contour knowledge transfer for salient object detection[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 355-370.
[19] ZHAO J X, LIU J J, FAN D P, et al. EGNet: edge guidance network for salient object detection[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, 2019: 8779-8788.
[20] WANG W G, ZHAO S Y, SHEN J B, et al. Salient object detection with pyramid attention and salient edges[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 1448-1457.
[21] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[22] PANG Y W, ZHAO X Q, XIANG T Z, et al. Zoom in and out: a mixed-scale triplet network for camouflaged object detection[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 2160-2170.
[23] WEI J, WANG S H, HUANG Q M. F3Net: fusion, feedback and focus for salient object detection[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020: 12321-12328.
[24] YANG C, ZHANG L H, LU H C, et al. Saliency detection via graph-nased manifold ranking[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013: 3166-3173.
[25] WANG L J, LU H C, WANG Y F, et al. Learning to detect salient objects with image-level supervision[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 136-145.
[26] LI G B, YU Y Z. Visual saliency based on multiscale deep features[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 5455-5463.
[27] YAN Q, XU L, SHI J P, et al. Hierarchical saliency detection[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013: 1155-1162.
[28] LI Y, HOU X D, KOCH C, et al. The secrets of salient object segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014: 280-287.
[29] EVERINGHAM M, VAN G L, WILLIAMS C K I, et al. The pascal visual object classes (VOC) challenge[J]. International Journal of Computer Vision, 2010, 88(2): 303-338.
[30] FAN D P, GONG C, CAO Y, et al. Enhanced-alignment measure for binary foreground map evaluation[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018: 698-704.
[31] CHEN S H, TAN X L, WANG B, et al. Reverse attention for salient object detection[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 234-250.
[32] LIU J J, HOU Q B, CHENG M M, et al. A simple pooling-based design for realtime salient object detection[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, 2019: 3912-3921.
[33] HOU Q B, ZHANG Z C, HUANG C Y, et al. BASNet: boundary-aware salient object detection[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, 2019: 7479-7489.
[34] ZHAO X Q, PANG Y W, ZHANG L H, et al. Suppress and balance: a simple gated network for salient object detection[C]//Proceedings of the 16th European Conference on Computer Vision, 2020: 35-51.
[35] JUN W, WANG S H, WU Z, et al. Label decoupling framework for salient object detection[C]//Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition, 2020: 13022-13031.
[36] LIU Y, ZHANG X Y, BIAN J W, et al. SAMNet: stereo-scopically attentive multi-scale network for lightweight salient object detection[J]. IEEE Transactions on Image Processing, 2021, 30(4): 3804-3814.
[37] ZHANG M, LIU T W, PIAO Y R, et al. Auto-MSFNet: search multi-scale fusion network for salient object detection[C]//Proceedings of the 29th ACM International Conference on Multimedia, 2021: 667-676.