Elsevier

Information Sciences

Volume 281, 10 October 2014, Pages 586-600
Information Sciences

Salient region detection for complex background images using integrated features

https://doi.org/10.1016/j.ins.2013.12.043Get rights and content

Abstract

We develop a novel algorithm for detecting salient regions. By analyzing the advantages and disadvantages of existing methods, five principles for designing salient region detection algorithms are summarized. Based on these principles, we propose a novel method that generates saliency map with highlighted salient regions by utilizing two different features, namely visual saliency value and spatial weight. The visual saliency value is determined based on local contrast differences and low-level feature frequencies. The spatial weight is computed by analyzing the size and locations of salient regions. Extensive experiments on benchmark image dataset demonstrate that the proposed algorithm outperforms seven state-of-the-art methods on complex background images.

Introduction

One common thread in the recent literature is the increasing in methods that simulate human biological systems, including neural network model [8], decision model [3], visual attention model [20], and so on. Visual attention results both from fast, unintentionally bottom-up visual saliency of the retinal input, as well as from slower, top-down memory and volition based processing that is task-dependent [2]. Since the bottom-up saliency does not require any prior knowledge and runs faster than the top-down saliency, it is widely applied in image segmentation [9], adaptive image display, image/video compression [25], [14], sketch retrieval, image retrieval [23], etc. We focus on the bottom-up saliency module in this paper.

In general, bottom-up saliency module can be classified into local and global methods. Local methods determine saliency value of pixels based on local contrast (i.e. center-to-surrounding difference) while global methods consider center-to-global differences or suppress frequently occurred features. For salient objects in simple background images, local contrast is equivalent to global contrast due to the equivalence relation between the center-to-surrounding difference and the center-to-global difference. As a result, both local and global methods work well on simple background images. As for the images with complex background, however, neither of them performs well. Moreover, several issues have been observed with the application of two methods (see Fig. 1):

  • Problem 1: Local methods highlight the boundaries more than the content of salient regions (red boxes in Fig. 1).

  • Problem 2: Global methods tend to undervalue big salient region or treat big salient region as background (blue boxes in Fig. 1).

  • Problem 3: In complex background image, the background usually includes various areas that are visually different. Local methods offer high saliency value to pixels on the border of these areas (green boxes in Fig. 1).

  • Problem 4: In complex background image, the background usually contains tiny patches with excessively infrequent visual features. As human perspective, they are too sparse to attract attention. Nonetheless, global methods may produce high saliency value to these noises (yellow boxes in Fig. 1).

To overcome these issues, we conclude five characteristics of salient regions in complex background images (see Table 1).

According to these principles, we propose a novel saliency detection method to solve the above problems. Our method consists of three parts:

  • 1.

    Local part: we detect pixels with high local visual contrast as source set based on Principle 1, then extend this set to candidate set based on Principle 2.

  • 2.

    Global part: we propose a novel measure to assign visual saliency values to candidates based on Principles 3 and 4.

  • 3.

    Spatial part: we propose a simple, fast algorithm which computes spatial weights based on the estimation of the smallest circumscribed circle containing all salient regions. This part follows Principle 5.

The final saliency value of pixels is considered to be the combination of visual saliency value and spatial weight.

The rest of the paper is organized as follows. Details of the proposed algorithm is presented and analyzed in Section 2. In Section 3, we perform extensive experiments to evaluate our method on two benchmark image sets. One image set consists of natural scene images with complex texture background. The other is one of the largest image sets with relatively simple background. Applications of our method in sketch retrieval are also introduced in Section 3. The related work is reviewed in Section 4 and concluding remarks are presented in Section 5.

Section snippets

Salient region detection using integrated features

In order to distinguish salient regions from complex background images, we propose a novel saliency detection method satisfying Principles (1)–(5). The framework of our method is shown in Fig. 2, and details of the framework are presented as below:

  • Local part (details in Section 2.1): In this part, we first generate source set which consists of pixels with high local visual contrast. Visual features of the source set are then used to generate candidate set, aiming at finding content pixels as

Experiment

In this section, we evaluate the effectiveness of the proposed approach. The experiment contains three parts. The first part evaluates the performance of the proposed approach by comparing with seven state-of-the-art saliency detection methods. The selection of comparing methods is refer to [7], [2]: citation (IT [17] is widely cited), variety (IT, AC [1], NS [22] are local methods, LC [28], RC [7], FT [2], SR [15] are global methods), recency (LC, SR, AC, FT, NS, RC are recent methods). The

Related work

In general, the bottom-up saliency detection approaches normally select saliency region on a basis of the unconscious stimulation by visual contrast.

Methods on the local contrast basis determine saliency regions by comparing them with surroundings. Itti et al. [17] used center-surrounding difference of intensity, color and orientation to yield feature maps, and then combined them into the final saliency map. It was the first practice with feature of integration to yield saliency map

Conclusion

This paper proposes five principles for saliency detection in complex background images. Based on these principles, a novel method using integrated features is proposed for detecting salient regions in complex background images. The integrated method computes saliency value of pixels by simultaneously considering pixel-to-neighborhood contrast, low-level feature frequencies and spatial coherence. The integrated method is simple, fast, and yields the same size saliency map with investigated

Acknowledgments

The work described in this paper was fully supported by the Grants from the National Nature Science Foundation of China (61271428, 61273247, 61303159); National Key Technology Research and Development Program of China (2012BAH39B02). This work was supported in part to Dr. Qi Tian by ARO grant W911NF-12-1-0057, NSF IIS 1052851, Faculty Research Awards by Google, FXPAL, and NEC Laboratories of America, and 2012 UTSA START-R Research Award respectively. This work was supported in part by NSFC

References (28)

  • P.F. Felzenszwalb et al.

    Efficient graph-based image segmentation

    Int. J. Comput. Vis.

    (2004)
  • S. Goferman et al.

    Context-aware saliency detection

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2012)
  • C. Guo et al.

    Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform

  • S. He, J. Han, X. Hu, M. Xu, L. Guo, T. Liu, A biologically inspired computational model for image saliency detection,...
  • Cited by (18)

    • MCCH: A novel convex hull prior based solution for saliency detection

      2019, Information Sciences
      Citation Excerpt :

      Visual saliency [13] is one of the classical ways to find regions of interest in the image. In the past many years, abundant efforts have been made on salient object detection [10,22,26,29,32,40,41,48,50]. The essence of the salient object detection is to highlight object(s) of interest in the image [22].

    • Automated 3D segmentation of brain tumor using visual saliency

      2018, Information Sciences
      Citation Excerpt :

      Such models seem to predict based on certain aspects of human eye movement [14]. Visual saliency can be defined as the outcome of comparing a region with its surrounding, with respect to unpredictability, contrast and rarity [1,25,26,38,43]. The comparison can be done in terms of low level feature similarity.

    • Complex networks driven salient region detection based on superpixel segmentation

      2017, Pattern Recognition
      Citation Excerpt :

      The implementation of region based methods can be realized in various ways, such as multi-dimension, histogram based contrast, color distribution, etc. By the way, some methods work on both local and global consideration, e.g., Goferman et al. [36], Zhang et al. [37]. Peng et al. [38] propose a saliency cut based on high-order energy term for stereo images.

    View all citing articles on Scopus
    View full text