Uncovering the temporal dynamics of scene understanding using Event-Related Potentials

• ¹ Wright State University, Department of Psychology, United States

Recognizing images of large-scale real-world environments is an essential skill in many professional contexts, ranging from defense and intelligence to driving and urban planning. However, relatively little is known about the temporal dynamics of scene recognition, and particularly its underlying neural temporal dynamics. Uncovering how scene processing unfolds in the brain over time (especially focusing on the earliest time windows in which the brain detects diagnostic scene information) is important not only for providing insights into the nature of scene understanding, but also for applied purposes. Prominent examples of such applications are the identification of specific real-time neural markers for the extraction of diagnostic scene information, as well as providing implicit measures for tracking down and evaluating image analyst training. In the present work, we describe a series of studies aimed at identifying and characterizing the functional significance of early electrophysiological markers of scene understanding. Our starting point for the current research is a recent surge in behavioral and neuroimaging studies that highlight the importance of diagnostic global scene properties (GSPs) for fast scene categorization (for a review, see Groen et al., 2017). Diagnostic GSPs, originally described by Greene and Oliva (2009), are mid-level image representations that convey information about the structure (“is it an open or a closed space?”) and function (“how easily can I navigate in this environment?”) of visual scenes. In our first study, we sought to establish EEG as a useful measure of scene diagnosticity by recording Event-Related Potentials (ERPs) from participants while they viewed naturalistic images of real-world scenes varying along two GSPs: spatial expanse (open/closed) and naturalness (manmade/natural). We found that these two GSPs modulated the amplitude of early sensory-evoked ERP components, particularly the posterior P2 (Harel et al., 2016), suggesting that information about the structure and function of visual scenes is encoded in the brain by 220ms post-stimulus onset. Given that open and closed scenes can be thought as two ends on a navigability continuum, we next reasoned that these ERP markers might contain information about the number of pathways that afford movement in the local environment. To test this idea, we recorded ERPs from participants in a second study while they viewed computer-generated room scenes used in a previous fMRI study of navigability (Bonner & Epstein, 2017). By simply changing the number of doors we were able to systematically control the number of movement paths in the scene, while keeping the overall size and shape of the environment constant. We found that rooms with no doors evoked a higher P2 response than rooms with three doors, analogous to the higher P2 amplitude to closed relative to open scenes. P2 amplitude to rooms with one or two doors was higher than three-door rooms but lower than the response to no-door rooms. This parametric navigability effect persisted following the P2 peak (~220ms), lasting up until 650ms post-stimulus onset. These results suggest that the perceived ease of navigation in a scene is represented in both early and late stages of scene perception, complementing fMRI research showing that the occipital place area automatically encodes the structure of navigable spaces. To further investigate the extent to which GSPs are indeed processed automatically (or rather influenced by their task-relevance), we conducted a third study in which we manipulated the task context under which the GSPs were perceived. Participants saw the same scenes as in the first study but now they were asked to actively categorize them, based either on their naturalness or spatial expense. We replicated previous results, and found that task context had very little impact on the early ERP responses to the naturalness and spatial expanse of the scenes: P2 amplitude could be used to distinguish between open and closed scenes even when participants were not actively attending to that dimension, and that also held for the distinction between manmade and natural scenes. These findings suggest that the extraction of global scene information is very little influenced by task-relevance, and supports the notion these early ERP markers of scene perception reflect automatic extraction of diagnostic scene information. In summary, the series of studies described here suggest that diagnostic global scene properties are processed robustly and automatically within the first 250 milliseconds of processing and can be used to index ecological behaviors such as potential navigability of a given visual environment. Potential implications and applications of these findings will be further discussed.