Saliency in images and video: a brief survey
Saliency in images and video: a brief survey
- Author(s): K. Duncan and S. Sarkar
- DOI: 10.1049/iet-cvi.2012.0032
For access to this article, please select a purchase option:
Buy article PDF
Buy Knowledge Pack
IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.
Thank you
Your recommendation has been sent to your librarian.
- Author(s): K. Duncan 1 and S. Sarkar 1
-
-
View affiliations
-
Affiliations:
1: Computer Science and Engineering Department, University of South Florida, Tampa, USA
-
Affiliations:
1: Computer Science and Engineering Department, University of South Florida, Tampa, USA
- Source:
Volume 6, Issue 6,
November 2012,
p.
514 – 523
DOI: 10.1049/iet-cvi.2012.0032 , Print ISSN 1751-9632, Online ISSN 1751-9640
Salient image regions permit non-uniform allocation of computational resources. The selection of a commensurate set of salient regions is often a step taken in the initial stages of many computer vision algorithms, thereby facilitating object recognition, visual search and image matching. In this study, the authors survey the role and advancement of saliency algorithms over the past decade. The authors first offer a concise introduction to saliency. Next, the authors present a summary of saliency literature cast into their respective categories then further differentiated by their domains, computational methods, features, context and use of scale. The authors then discuss the achievements and limitations of the current state of the art. This information is augmented by an outline of the datasets and performance measures utilised as well as the computational techniques pervasive in the literature.
Inspec keywords: computer vision; object recognition; image matching; video signal processing
Other keywords:
Subjects: Optical, image and video signal processing; Video signal processing; Computer vision and image processing techniques
References
-
-
1)
- Heitger, F., von der Heydt, R., Kubler, O.: `A computational model of neural contour processing: figure-ground segregation and illusory contours', From Perception to Action Conf., 1994, September 1994, p. 181–192.
-
2)
- Li, Y., Zhou, Y., Xu, L., Yang, X., Yang, J.: `Incremental sparse saliency detection', IEEE Int. Conf. on Image Processing (ICIP), 7–10 November 2009, p. 3093–3096.
-
3)
- T. Kadir , M. Brady . Saliency, scale, and image description. Int. J. Comput. Vis. , 83 - 105
-
4)
- The PASCAL Visual Object Classes Homepage. http://pascallin.ecs.soton.ac.uk/challenges/VOC/.
-
5)
- Rapantzikos, K., Avrithis, Y., Kollias, S.: `Dense saliency-based spatiotemporal feature points for action recognition', IEEE Conf. on Computer Vision and Pattern Recognition, June 2009, p. 1454–1461.
-
6)
- Escalera, S., Radeva, P., Pujol, O.: `Complex salient regions for computer vision problems', IEEE Conf. on Computer Vision and Pattern Recognition, 17–22 June 2007, p. 1–8.
-
7)
- Duncan, K.: `Relational entropy-based measure of saliency', 2010, Master's, University of South Florida.
-
8)
- Gao, D., Vasconcelos, N.: `Integrated learning of saliency, complex features, and object detectors from cluttered scenes', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005, 2, p. 282–287, (20–25).
-
9)
- P.J. Burt , E.H. Adelson . The Laplacian Pyramid as a compact image code. IEEE Trans. Commun. , 4 , 532 - 540
-
10)
- Liu, T., Sun, J., Zheng, N.n., Tang, X., Shum, H.y.: `Learning to detect a salient object', IEEE Conf. on Computer and Vision Pattern Recognition, 2007, p. 1–8.
-
11)
- Michael Bileschi, S.: `StreetScenes: towards scene understanding in still images', 2006, PhD, Massachusetts Institute of Technology.
-
12)
- Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PAS- CAL Visual Object Classes Challenge 2009 (VOC2009). http://www.pascal-network.org/challenges/VOC/voc2009/workshop/index.html.
-
13)
- A. Torralba . Modeling global scene factors in attention. J. Opt. Soc. Am. , 1407 - 1418
-
14)
- N.D.B. Bruce , J.K. Tsotsos . Saliency, attention, and visual search: an information theoretic approach. J. Vis. , 1 - 24
-
15)
- J. Ponce , T.L. Berg , M. Everingham . (2006) Dataset issues in object recognition.
-
16)
- G. Heidemann . The long-range saliency of edge and corner-based salient points. IEEE Trans. Image Process. , 11 , 1701 - 1706
-
17)
- M. de Brecht , J. Saiki . A neural network implementation of a saliency map model. Neural Netw. , 10 , 1467 - 1474
-
18)
- T.F. Syeda-Mahmood . Detecting perceptually salient texture regions in images. Comput. Vis. Image Underst. , 93 - 108
-
19)
- T. Avraham , M. Lindenbaum . Esaliency (extended saliency): meaningful attention using stochastic image modeling. IEEE Trans. Pattern Anal. Mach. Intell. , 4 , 693 - 708
-
20)
- Li, F.-F., Andreeto, M., Ranzato, M.A.: Caltech 101. http://www.vision.caltech.edu/Image_Datasets/Caltech101.
-
21)
- Guo, C., Ma, Q., Zhang, L.: `Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform', IEEE Conf. on Computer Vision and Pattern Recognition, June 2008, p. 1–8.
-
22)
- G. Zhu , Y. Zheng , D. Doermann , S. Jaeger . Signature detection and matching for document image retrieval. IEEE Trans. Pattern Anal. Artif. Intell. , 2015 - 2031
-
23)
- X. Chen , G.J. Zelinsky . Real-world visual search is dominated by top-down guidance. Vis. Res. , 24 , 4118 - 4133
-
24)
- Griffin, G., Holub, A., Perona, P.: `Caltech-256 object category dataset', Technical report 7694, 2007.
-
25)
- D. Walther , C. Koch . Modeling attention to salient proto-objects. Neural Netw. , 9 , 1395 - 1407
-
26)
- T. Liu , Z. Yuan , J. Sun . Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. , 2 , 1 - 15
-
27)
- University of Washington Ground Truth Database. http://www.cs.washington.edu/research/imagedatabase/groundtruth/.
-
28)
- Laptev, I., Marsza lek, M., Schmid, C., Rozenfeld, B.: `Learning realistic human actions from movies', IEEE Conf. on Computer Vision and Pattern Recognition, 2008.
-
29)
- Pierrard, J.-S., Vetter, T.: `Skin detail analysis for face recognition', IEEE Conf. on Computer Vision and Pattern Recognition, 17–22 June 2007, p. 1–8.
-
30)
- A.J. Bell , T.J. Sejnowski . The independent components of natural scenes are edge filters. Vis. Res. , 3327 - 3338
-
31)
- Fei-Fei, L., Fergus, R., Perona, P.: `Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories', IEEE Conf. on Computer Vision and Pattern Recognition, 2004.
-
32)
- L. Itti , C. Koch . A saliency-based search mechanism for overt and covert shifts of visual attention. Vis. Res. , 1489 - 1506
-
33)
- Peters, R.J., Itti, L.: `Beyond bottom-up: incorporating task-dependent influences into a computational model of spatial attention', IEEE Conf. on Computer Vision and Pattern Recognition, 2007, p. 1–8.
-
34)
- K. Mikolajczyk , T. Tuytelaars , C. Schmid , A. Zisserman , J. Matas , F. Schaffslatzky , T. Kadir , V.L. Gool . A comparison of affine region detectors. Int. J. Comput. Vis. , 1 , 43 - 72
-
35)
- Carson, C., Belongie, S., Greenspan, H., Malik, J.: `Region-based image querying', IEEE Workshop on content-based access of image and video libraries, 1997, 20, p. 42–49.
-
36)
- Recognition of Human Actions. http://www.nada.kth.se/cvap/actions/.
-
37)
- N. Ouerhani , R. von Hartburg , H. Hugli , R. Muri . Empirical validation of the saliency-based model of visual attention. Electron. Lett. Comput. Vis. Image Anal. , 13 - 24
-
38)
- Gao, D., Vasconcelos, N.: `Discriminant saliency for visual recognition from cluttered scenes', Neural Information Processing Systems, June 2009, p. 481–488.
-
39)
- J. Harel , C. Koch , P. Perona . Graph-based visual saliency. Adv. Neural Inf. Process. Syst. , 545 - 552
-
40)
- M. Everingham , A. Zisserman , C.K.I. Williams , L. Van Gool . The PASCAL Visual Object Classes Challenge 2006 (VOC2006) Results.
-
41)
- J. Sun . Microsoft Research Asia Salient Object Database.
-
42)
- Hare, J.S., Lewis, P.H.: `Scale saliency: applications in visual matching, tracking and view-based object recognition', Distributed Multimedia Systems 2003 Visual Information Systems 2003, 2003, p. 436–440.
-
43)
- B. Chalmond , B. Francesconi , S. Herbin . Using hidden scale for salient object detection. IEEE Trans. Image Process. , 9 , 2644 - 2656
-
44)
- Stottinger, J., Hanbury, A., Gevers, T., Sebe, N.: `Lonely but attractive: sparse color salient points for object retrieval and categorization', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, June 2009, p. 1–8.
-
45)
- D. Gao , S. Han , N. Vasconcelos . Discriminant saliency, the detection of suspicious coincidences, and applications to visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. , 989 - 1005
-
46)
- B. Schiele , J.L. Crowley . Recognition without correspondence using multidimensional receptive field histograms. Int. J. Comput. Vis. , 31 - 50
-
47)
- Hou, X., Zhang, L.: `Saliency detection: a spectral residual approach', IEEE Conf. on Computer Vision and Pattern Recognition, 2007, p. 1–8.
-
48)
- Liu, T., Zheng, N., Ding, W., Yuan, Z.: `Video attention: learning to detect a salient object sequence', Int. Conf. on Pattern Recognition, 8–11 December 2008, p. 1–4.
-
49)
- C.G. Healey , K.S. Booth , J.-T. Enns . High-speed visual estimation using peattentive processing. ACM Trans. Comput.-Hum. Interact. , 107 - 135
-
50)
- Nene, S.A., Nayar, S.K., Murase, H.: `Columbia object image library (COIL-20)', Technical report, 1996.
-
51)
- E. Vazquez , T. Gevers , M. Lucassen , J. van de Weijer , R. Baldrich . Saliency of color image derivatives: a comparison between computational models and human perception. J. Opt. Soc. Am. , 3 , 613 - 621
-
52)
- Gao, D., Vasconcelos, N.: `Bottom-up saliency is a discriminant process', IEEE 11th Int. Conf. on Computer Vision, October 2007, p. 1–6.
-
53)
- S. Marat , T.H. Phuoc , L. Granjon , N. Guyader , D. Pellerin , A. Guérin-Dugué . Modelling spatio-temporal saliency to predict gaze direction for short videos. Int. J. Comput. Vis. , 3 , 231 - 243
-
54)
- Itti, L., Baldi, P.: `A principled approach to detecting surprising events in video', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, June 2005, 1, p. 631–637.
-
55)
- Zhang, Q., Xiao, H.: `Biologically motivated salient regions detection approach', Second Int. Symp. on Intelligent Information Technology Application, 20–22 December 2008, 2, p. 1100–1104.
-
56)
- V. Gopalakrishnan , Y. Hu , D. Rajan . Salient region detection by modeling distributions of color and orientation. IEEE Trans. Multimedia , 5 , 892 - 905
-
57)
- Tobacco800 Complex Document Image Database and Groundtruth. http://www.umiacs.umd.edu/~zhugy/Tobacco800.html.
-
58)
- N. Bruce , J. Tsotsos . Saliency based on information maximization. Adv. Neural Inf. Process. Syst. , 155 - 162
-
59)
- Cai, J.-Z., Zhang, M.-X., Chang, J.-Y.: `A novel salient region extraction based on color and texture features', Int. Conf. on Wavelet Analysis and Pattern Recognition, 8–15 2009, p. 12–15.
-
60)
- Y.-M. Jang , S.-W. Ban , M. Lee . Stereo saliency map considering affective factors in a dynamic environment. Neural Inf. Process. , 1055 - 1064
-
61)
- J. Maver . Self-similarity and points of interest. IEEE Trans. Pattern Anal. Mach. Intell. , 7 , 1211 - 1226
-
62)
- C. Guo , L. Zhang . A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans. Image Process. , 1 , 185 - 198
-
63)
- Judd, T., Ehinger, K., Durand, F., Torralba, A.: `Learning to predict where humans look', Int. Conf. on Computer Vision, September 2009, p. 2106–2113.
-
64)
- Fu, Y., Cheng, J., Li, Z., Lu, H.: `Saliency cuts: an automatic approach to object segmentation', Int. Conf. on Pattern Recognition, December 2008, p. 1–4.
-
65)
- S. Mahamud , L.R. Williams , K.K. Thornber , K. Xu . Segmentation of multiple salient closed contours from real images. IEEE Trans. Pattern Anal. Mach. Intell. , 4 , 433 - 444
-
66)
- Seo, H.-J., Milanfar, P.: `Visual saliency for automatic target detection, boundary detection, and image quality assessment', IEEE Int. Conf. on Acoustics Speech and Signal Processing, 14–19 March 2010, p. 5578–5581.
-
67)
- Sha'ashua, A., Ullman, S.: `Structural saliency: the detection of globally salient structures using a locally connected network', Int. Conf. on Computer Vision, December 1988, p. 321–327.
-
68)
- L. Itti , C. Gold , C. Koch . Visual attention and target detection in cluttered natural scenes. Opt. Eng. , 9 , 1784 - 1793
-
69)
- Chen, H.-Y., Leou, J.-J.: `A new visual attention model using texture and object features', IEEE Eighth Int. Conf. on Computer and Information Technology Workshops, 8–11 July 2008, p. 374–378.
-
70)
- V. Mahadevan , N. Vasconcelos . Spatiotemporal saliency in dynamic scenes. IEEE Trans. Pattern Anal. Mach. Intell. , 1 , 171 - 177
-
71)
- Zhu, G., Zheng, Y., Doermann, D., Jaeger, S.: `Multi-scale structural saliency for signature detection', IEEE Conf. on Computer Vision and Pattern Recognition, 17–22 June 2007, p. 1–8.
-
72)
- Hollywood Human Actions Dataset. http://www.irisa.fr/vista/Equipe/People/Laptev/download.html.
-
73)
- T. Lindeberg . Scale-space theory: a basic tool for analyzing structures at different scales. J. Appl. Stat. , 225 - 270
-
74)
- A. Berengolts , M. Lindenbaum . On the distribution of saliency. IEEE Trans. Pattern Anal. Mach. Intell. , 1973 - 1987
-
75)
- Martin, D., Fowlkes, C., Tal, D., Malik, J.: `A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics', Int. Conf. Computer Vision, 2001, p. 416–423.
-
76)
- B.W. Tatler , R.J. Baddeley , I.D. Gilchrist . Visual correlates of fixation selection: effects of scale and time. Vis. Res. , 5 , 643 - 659
-
77)
- Navalpakkam, V., Itti, L.: `An integrated model of top-down and bottom-up attention for optimizing detection speed', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2006, p. 2049–2056.
-
78)
- Lewis, D., Agam, G., Argamon, S., Frieder, O., Grossman, D., Heard, J.: `Building a test collection for complex document information processing', 29thAnnual Int. ACM SIGIR Conf., 2006, p. 665–666.
-
1)