Skip to main content
SpringerLink
Log in

If Visual Saliency Predicts Search, Then Why? Evidence from Normal and Gaze-Contingent Search Tasks in Natural Scenes

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

The Itti and Koch (Vision Research 40: 1489–1506, 2000) saliency map model has inspired a wealth of research testing the claim that bottom-up saliency determines the placement of eye fixations in natural scenes. Although saliency seems to correlate with (although not necessarily cause) fixation in free-viewing or encoding tasks, it has been suggested that visual saliency can be overridden in a search task, with saccades being planned on the basis of target features, rather than being captured by saliency. Here, we find that target regions of a scene that are salient according to this model are found quicker than control regions (Experiment 1). However, this does not seem to be altered by filtering features in the periphery using a gaze-contingent display (Experiment 2), and a deeper analysis of the eye movements made suggests that the saliency effect is instead due to the meaning of the scene regions. Experiment 3 supports this interpretation, showing that scene inversion reduces the saliency effect. These results suggest that saliency effects on search may have nothing to do with bottom-up saccade guidance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Buswell GT. How people look at pictures: a study of the psychology of perception in art. Chicago: University of Chicago Press; 1935.

    Google Scholar 

  2. Yarbus AL. Eye movements and vision. New York: Plenum; 1967.

    Google Scholar 

  3. Itti L, Koch C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vis Res. 2000;40(10–12):1489–506.

    Article  PubMed  CAS  Google Scholar 

  4. Treisman A, Gelade G. A feature-integration theory of attention. Cogn Psychol. 1980;12:97–136.

    Article  PubMed  CAS  Google Scholar 

  5. Koch C, Ullman S. Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiol. 1985;4:219–27.

    CAS  Google Scholar 

  6. Mannan S, Ruddock K, Wooding D. The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images. Spat Vis. 1996;10(10):65–188.

    Google Scholar 

  7. Reinagel P, Zador AM. Natural scene statistics at the centre of gaze. Network-Computation In Neural Systems. 1999;10(4):341–50.

    Article  CAS  Google Scholar 

  8. Tatler BW, Baddeley RJ, Gilchrist ID. Visual correlates of fixation selection: effects of scale and time. Vis Res. 2005;45(5):643–59.

    Article  PubMed  Google Scholar 

  9. Itti L, Koch C. Computational modelling of visual attention. Nat Rev Neurosci. 2001;2(3):194–203.

    Article  PubMed  CAS  Google Scholar 

  10. Parkhurst D, Law K, Niebur E. Modeling the role of salience in the allocation of overt visual attention. Vis Res. 2002;42(1):107–23.

    Article  PubMed  Google Scholar 

  11. Peters RJ, et al. Components of bottom-up gaze allocation in natural images. Vis Res. 2005;45(18):2397–416.

    Article  PubMed  Google Scholar 

  12. Henderson JM, et al. Visual saliency does not account for eye movements during visual search in real-world scenes. In: van Gompel R, et al., editors. Eye movements: a window on mind and brain. Amsterdam: Elsevier; 2007. p. 537–62.

    Google Scholar 

  13. Foulsham T, Underwood G. What can saliency models predict about eye movements?. Spatial and sequential aspects of fixations during encoding and recognition. J Vis. 2008;8(6):1–17.

    Article  PubMed  Google Scholar 

  14. Harding G, Bloj M. Real and predicted influence of image manipulations on eye movements during scene recognition. J Vis. 2010;10(2):8.1–17.

    Google Scholar 

  15. Underwood G, Foulsham T, Humphrey K. Saliency and scan patterns in the inspection of real-world scenes: eye movements during encoding and recognition. Vis Cogn. 2009;17(6–7):812–34.

    Article  Google Scholar 

  16. Itti L. Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Vis Cogn. 2005;12(6):1093–123.

    Article  Google Scholar 

  17. Dorr M, Gegenfurtner KR, Barth E. The contribution of low-level features at the centre of gaze to saccade target selection. Vis Res. 2009;49(24):2918–26.

    Article  PubMed  Google Scholar 

  18. Underwood G. Cognitive processes in eye guidance: algorithms for attention in image processing. Cogn Comput. 2009;1:64–76.

    Article  Google Scholar 

  19. Underwood G, et al. Eye movements during scene inspection: a test of the saliency map hypothesis. Eur J Cogn Psychol. 2006;18(3):321–42.

    Article  Google Scholar 

  20. Foulsham T, Underwood G. How does the purpose of inspection influence the potency of visual saliency in scene perception? Perception. 2007;36:1123–38.

    Article  PubMed  Google Scholar 

  21. Henderson JM, Malcolm GL, Schandl C. Searching in the dark: cognitive relevance drives attention in real-world scenes. Psychon Bull Rev. 2009;16(5):850–6.

    Article  PubMed  Google Scholar 

  22. Chen X, Zelinsky GJ. Real-world visual search is dominated by top-down guidance. Vis Res. 2006;46(24):4118–33.

    Article  PubMed  Google Scholar 

  23. Foulsham T, Underwood G. Does conspicuity enhance distraction? Saliency and eye landing position when searching for objects. Quart J Exp Psychol. 2009;62(6):1088–98.

    Article  Google Scholar 

  24. Navalpakkam V, Itti L. Modeling the influence of task on attention. Vis Res. 2005;45(2):205–31.

    Article  PubMed  Google Scholar 

  25. Torralba A, et al. Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychol Rev. 2006;113(4):766–86.

    Article  PubMed  Google Scholar 

  26. Cutsuridis V. A cognitive model of saliency, attention, and picture scanning. Cogn Comput. 2009;1:292–9.

    Article  Google Scholar 

  27. McConkie GW, Rayner K. Span of effective stimulus during a fixation in reading. Percept Psychophys. 1975;17(6):578–86.

    Article  Google Scholar 

  28. Rayner K. Eye movements in reading and information processing: 20 years of research. Psychol Bull. 1998;124(3):372–422.

    Article  PubMed  CAS  Google Scholar 

  29. Geisler WS, Perry JS, Najemnik J. Visual search: the role of peripheral information measured using gaze-contingent displays. J Vis. 2006;6(9):858–73.

    Article  PubMed  Google Scholar 

  30. Castelhano MS, Henderson JM. Initial scene representations facilitate eye movement guidance in visual search. J Exp Psychol Hum Percept Perform. 2007;33(4):753–63.

    Article  PubMed  Google Scholar 

  31. Loschky LC, McConkie GW. Investigating spatial vision and dynamic attentional selection using a gaze-contingent multiresolutional display. J Exp Psychol Appl. 2002;8(2):99–117.

    Article  PubMed  Google Scholar 

  32. Foulsham T, Teszka R, Kingstone A. Saccade control in natural images is shaped by the information visible at fixation: evidence from asymmetric gaze-contingent windows. Atten Percept Psychophys. (in press).

  33. Valentine T. Upside-down faces—a review of the effect of inversion upon face recognition. Br J Psychol. 1988;79:471–91.

    PubMed  Google Scholar 

  34. Gauthier I, Tarr MJ. Becoming a ‘‘greeble’’ expert: exploring mechanisms for face recognition. Vis Res. 1997;37(12):1673–82.

    Article  PubMed  CAS  Google Scholar 

  35. Husk JS, Bennett PJ, Sekuler AB. Inverting houses and textures: Investigating the characteristics of learned inversion effects. Vis Res. 2007;47(9):3350–9.

    Article  PubMed  Google Scholar 

  36. Kelley TA, Chun MM, Chua KP. Effects of scene inversion on change detection of targets matched for visual salience. J Vis. 2003;3(1):1–5.

    Article  PubMed  Google Scholar 

  37. Loftus GR, Mackworth NH. Cognitive determinants of fixation location during picture viewing. J Exp Psychol Hum Percept Perform. 1978;4(4):565–72.

    Article  PubMed  CAS  Google Scholar 

  38. Henderson JM, Weeks PA, Hollingworth A. The effects of semantic consistency on eye movements during complex scene viewing. J Exp Psychol Hum Percept Perform. 1999;25(1):210–28.

    Article  Google Scholar 

  39. Underwood G, Foulsham T. Visual saliency and semantic incongruency influence eye movements when inspecting pictures. Quart J Exp Psychol. 2006;59(11):1931–49.

    Article  Google Scholar 

  40. van Diepen PMJ, d’Ydewalle G. Early peripheral and foveal processing in fixations during scene perception. Visual Cognition. 2003;10(1):79–100.

    Article  Google Scholar 

  41. Henderson JM. Eye-movement control during visual object processing—effects of initial fixation position and semantic constraint. Can J Exp Psychol. 1993;47(1):79–98.

    PubMed  CAS  Google Scholar 

Download references

Acknowledgments

TF is supported by a Commonwealth Fellowship from the Government of Canada. GJU was supported by project grant EP/E006329/1 from the EPSRC (UK). We are also grateful to Laurent Itti et al. for making the saliency map model available, and the comments of an anonymous reviewer.

Author information

Authors and Affiliations

  1. Department of Psychology, University of British Columbia, 2136 West Mall, Vancouver, BC, V6T 1Z4, Canada

    Tom Foulsham

  2. School of Psychology, University of Nottingham, Nottingham, NG7 2RD, UK

    Geoffrey Underwood

Authors
  1. Tom Foulsham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Geoffrey Underwood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tom Foulsham.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Foulsham, T., Underwood, G. If Visual Saliency Predicts Search, Then Why? Evidence from Normal and Gaze-Contingent Search Tasks in Natural Scenes. Cogn Comput 3, 48–63 (2011). https://doi.org/10.1007/s12559-010-9069-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-010-9069-9

Keywords

Search

Navigation