(How) observed eye-contact modulates gaze following. An fMRI study

Humans are highly sensitive to directional gaze cues and rapidly shift attention in accordance with others’ gaze (i.e., gaze following). Besides providing information about the physical environment, for instance, the location of an object, gaze direction can be used to extract information about the social environment, such as whether or not two people are interacting with each other. In the present fMRI study we investigated how these two different types of information conveyed by gaze direction interact with one another. Participants saw two faces that were either looking at each other or away from each other before jointly shifting gaze toward one of two target locations. Targets either appeared at the gazed at or the non-gazed at location. Behaviorally, gaze following (faster responses to congruent versus incongruent trials) was more prominent after observing eye contact than after observing no eye contact. In line with behavioral findings, neuroimaging results revealed enhanced activation in fronto-parietal and temporal areas in congruent trials when faces had looked at each other versus away from each other. These findings demonstrate that observing an attentional relation between others augments processing of their subsequent gaze cues.

Recent findings from a gaze-cueing paradigm revealed that eye contact can enhance gaze following even when it is merely observed between others (Böckler, Knoblich, & Sebanz, 2011). When participants viewed pictures of faces that were either looking at each other or away from each other (see Figure 1), they followed gaze more when it was provided by faces that had previously established eye contact with each other. Hence, even though eye contact that is shared between others does not indicate a direct communicative intent toward the viewer, it may still function as a social signal concerning the importance of a subsequent action, thereby modulating the processing of gaze cues.
The present study aimed at addressing the neural underpinnings of this "observed eye-contact effect." Previous behavioral studies revealed a strong gaze cue effect after the gazers had established eye contact with each other, while no (or a reduced) gaze congruency effect was found when faces had been looking away from each other (Böckler et al., 2011;Böckler, Timmermans, Sebanz, Vogeley, & Schilbach, 2014). What this behavioral pattern does not disclose, however, is whether this interaction is due to a processing benefit for gaze-congruent trials after the gazers had looked at each other, or whether it is due to a reduced cost in incongruent trials when faces had looked away. Hence, the question is how processing of congruent (and incongruent) gaze cues after observing eye contact differs from processing of these cues after observing the faces looking away from each other. We hypothesized an increase of activation in typical fronto-parietal and temporal attention areas in congruent trials after observed eye contact (as opposed to averted gaze), which would suggest a processing benefit of congruent gaze cues Figure 1. Sequence of events on each trial. After looking at each other or away from each other, the two faces simultaneously shifted their gaze to a location that was either congruent or incongruent with the target location. after observing eye contact. Alternatively, if observing averted gaze reduces the processing cost for incongruent trials, we should find reduced activation in the above mentioned attention areas in incongruent trials in the averted gaze compared to the eye contact condition. In order to replicate the behavioral findings and to investigate the neural underpinnings, the original paradigm (Böckler et al., 2011) was adapted to an fMRI setting.

Participants
Twenty-one students (mean age 24.6 years, 13 female) participated in the experiment and received payment or course credits for participation. All of them were right-handed and had normal or corrected to normal vision. Data from an additional six participants were excluded due to technical problems or problems during data acquisition. The study was approved by the ethics committee of the Radboud University, Nijmegen and the Donders Institute for Brain, Cognition, and Behaviour and complied with the Declaration of Helsinki, and all participants signed informed consent prior to participation.

Stimuli and design
We implemented a 2 (attentional relation: Observed eye contact vs. observed averted gaze) x 2 (gaze cue: Congruent vs. incongruent) design. Stimuli consisted of photographs of horizontally aligned female faces and targets were photographs of an apple and a pear. Each trial comprised three stages: (1) an initial phase during which the attentional relation between the observed agents was established, (2) an attention direction phase, and (3) the presentation of the target. In the initial phase (1), the faces were presented at the center of the screen (see Figure 1), either looking at each other (eye contact) or away from each other (averted gaze). In the attention direction phase (2), the faces simultaneously gazed up or down. In the target presentation phase (3), the target (apple or pear) appeared either at the gazed-at location (congruent gaze cue) or at the not gazed-at location (incongruent gaze cue).

Procedure
After signing informed consent, participants were instructed about the scanning procedure and were familiarized with the task by means of a training session outside the scanner (16 trials, four trials for each condition). The experiment consisted of a total of eight null-event trials (in which the screen stayed blank) and 128 experimental trials. The four conditions (two attentional relations and congruent vs. incongruent gaze cues) were randomized across the experiment. Participants were instructed to respond to the identity of the target by pressing one of two response buttons with their right hand (twochoice task; e.g., press index finger button for apple and middle finger button for pear). Each trial started with the central presentation of the two faces, either looking at or away from each other for 3000-5000 ms. A fixation cross was shown between the two faces throughout the trial. Subsequently, both faces looked at one of the two target locations (up or down). This picture stayed on the screen and following stimulus onset asynchronies (SOAs) of 500, 600, or 700 ms, a picture of an apple or a pear was presented additionally at one of the locations until participants responded (up to 2000 ms). Intertrial intervals (ITIs) (displaying a blank screen) lasted 4000-6000 ms. Response buttons were aligned orthogonally to target positions in order to exclude effects of stimulus-response compatibility. Conditions were randomized within blocks. The assignment of targets to responses was counterbalanced across subjects. Stimuli were presented with Presentation ® software (version 15.1).
Neuroimaging data analysis fMRI data were analyzed in an event-related design within the general linear model, using SPM8 software and Matlab R2007b. Prior to the analysis, the first six volumes were excluded to control for T1 equilibration effects. Six movement parameters (three translations and three rotations) were extracted from the first echo of each volume and subsequently used to correct for small head movements in all five echoes of each volume. Subsequently, all five echoes were combined into a single volume using a weighted average. To correct for delays in slice timing during image acquisition, the time courses of each voxel were realigned toward slice 16. After segmentation into gray and white matter, images were normalized to a standard EPI template within MNI space and resampled at an isotropic voxel size of 3 mm. Lastly, the images were convolved with a Gaussian smoothing kernel with 6 mm full width at half maximum (FWHM; Friston et al., 1995).
Two events of interest were used for the statistical analyses: The first frame of a trial (depiction of attentional relation: Eye contact vs. averted gaze, see Figure 1) and the third frame of a trial (target appears at congruent or incongruent location). Two onset regressors were created for the first event (eye contact vs. averted gaze) and four onset regressors for the four conditions in the second event (two attentional relations × two gaze cues). All regressors were convolved with a canonical hemodynamic response function (HRF). To correct for motionrelated artifacts, we modeled subject-specific realignment parameters as covariates of no interest. Linear contrasts of regression coefficients were computed at the individual subject level and then taken to a group-level random-effects analysis. The group analysis consisted of a one-sample t-test across the contrast images of all participants that indicated whether observed differences between conditions were significantly distinct from zero.
In order to investigate the processing of observed eye contact and observed averted gaze, the following contrasts were calculated (main effects of the attentional relation): Eye Contact > Averted Gaze (EC > AG) and Averted Gaze > Eye Contact (EC > AG). Main effects of gaze congruency were addressed with the contrasts Congruent Gaze Cues > Incongruent Gaze Cues (Cong > Incong) and Incongruent Gaze Cues > Congruent Gaze Cues (Inong > Cong). To investigate the effects of attentional relation in the context of both gaze-cue conditions (according to our main aim), the following specific contrasts were calculated: EC cong > AG cong , AG cong > EC cong , EC incong > AG incong , and AG incong > EC incong. To further address the effect of gaze congruency in the context of previous attentional relations we calculated the contrasts EC cong > EC incong , EC incong > EC cong , AG cong > AG incong , and AG incong > AG cong.
To protect against false-positive activations a double threshold was applied, by which only regions with a z-score exceeding 2.58 (p < .005, uncorrected) and a volume exceeding 297 mm 3 (k > 60) were considered (corresponding to p < .05, corrected) (Forman et al., 1995). This was determined in a Monte Carlo simulation using a Matlab script provided by Scott Slotnick (http://www2.bc.edu/~slot nics/scripts.htm).

Behavioral results
Reaction times (RTs) of trials with correct responses were analyzed by means of a repeated measures ANOVA including the within-subject factors attentional relation (eye contact vs. averted gaze) and gaze cue (congruent vs. incongruent). Reaction times were significantly faster for congruent than incongruent

Main effects attentional relation
The contrast Eye Contact > Averted Gaze (EC > AG) revealed significant suprathreshold activations (p < .005, k > 60) in right lingual gyrus (see Figure 3 and Table 1). The reverse contrast (AG > EC) yielded bilateral activation in insula, activation in right anterior cingulate cortex, right middle frontal gyrus, left inferior frontal sulcus, left precentral gyrus, left middle and superior orbital gyrus, right lingual gyrus, right superior parietal lobule, and left middle temporal gyrus and temporal pole (see Figure 3 and Table 1).

Main effects gaze cue
The main effect of gaze congruency revealed activation in left temporal lobe and right superior temporal gyrus for the contrast Congruent Gaze Cues > Incongruent Gaze Cues (Cong > Incong). The opposite contrast (Incong > Cong) yielded bilateral activation in frontal eye fields, in middle frontal gyri (precentral and premotor areas), as well as middle occipital gyri bilaterally and left inferior parietal lobe (see Table 2).

Interaction attentional relation and gaze cue
The contrast EC cong > AG cong yielded significant activation in left inferior parietal lobe, left middle  temporal sulcus, and right precentral sulcus. No suprathreshold activation was found for the contrasts AG cong > EC cong , EC incong > AG incong , and AG incong > EC incong. (see Figure 4 and Table 3). Suprathreshold activation was revealed for AG incong > AG cong ; this contrast was related to bilateral activation in middle frontal gyri (frontal eye fields), left inferior frontal sulcus, left inferior and superior parietal lobe, and right middle occipital gyrus (see Figure 4 and Table 3). No significant activation was revealed for EC cong > EC incong , EC incong > EC cong , and AG cong > AG incong .

DISCUSSION
The present study aimed at investigating the influence of an observed attentional relation (i.e., eye contact) between others on the subsequent processing of their gaze. Behavioral findings replicated previous studies (Böckler et al., 2011(Böckler et al., , 2014 showing that gaze following was enhanced after gazers had established eye contact with each other. This demonstrates that, even though gaze following is a robust effect, it can be modulated by social signals including not only experienced but also observed eye contact.

Gaze congruency
In line with the neuroimaging literature on gaze following, we found enhanced activation in a wide range of areas in the fronto-parietal attention network (Corbetta, Patel, & Shulman, 2008), including middle frontal gyri bilaterally, right precentral gyrus, and left inferior parietal lobe, as well as middle occipital gyri bilaterally when comparing incongruent with congruent gaze cues (   is elicited by the target appearing at the incongruent (i.e., not gazed at) location: When gazes provided spatial information that differed from the exogenous spatial information delivered by the target, additional attention shifts (e.g., reallocation of attention) were prompted. Based on the data at hand, we cannot conclude whether attentional shifts were entirely covert. Future studies aiming at disentangling the contribution of overt eye movements and covert attention shifts would benefit from tracking eye movements in the scanner. The opposite contrast (congruent vs. incongruent gaze cues) yielded less activation overall, which is in line with the literature (Schilbach et al., 2010). Activation was found in right superior temporal gyrus and left temporal pole. Both areas have been ascribed functions in gaze processing and social attention ( Table 2) (Nummenmaa & Calder, 2009;Williams et al., 2005) and processes like goal attribution in action observation or, more broadly speaking, integrating observed movement into context (Saxe, 2006;Schultz, Imamizu, Kawato, & Frith, 2004). Hence, in contrast to incongruent gaze cues, which triggered spatial attention shifts, congruent gazes toward the target elicited processes that may be more related to the interpretation of the visual scene (two people looking at the target together). Most importantly, both main contrasts of gaze cues replicate earlier findings (Bristow et al., 2007;Dichter & Belger, 2007;Pelphrey et al., 2003;Schilbach et al., 2010) and-together with our behavioral findings-indicate that the present paradigm successfully induced gaze following.

Observed attentional relation
Viewing two faces establishing eye contact (vs. looking away from each other) revealed activation in right lingual and fusiform gyrus. A very similar area has been reported for the processing of direct gaze (i.e., gaze at the participant) versus averted gaze (Calder et al., 2002;George et al., 2001) and seems to underlie the processing of configural face information. Unlike findings for experiencing direct versus averted gaze (Senju & Johnson, 2009), the mere observation of eye contact between others in the present study did not seem to activate the areas involved, for instance, in theory of mind or affective processing. It may be that participants perceived the two faces as engaged in a meaningful interaction irrespective of the momentary gaze direction, possibly because the same two faces were seen throughout the experiment. Studies showing enhanced activation of social brain areas when being engaged in direct gaze have typically used different faces (e.g., Calder et al., 2002). Of course, conclusive evidence on whether and how eye contact is processed differently when experienced than when observed can only come from studies that directly compare experienced and observed eye contact.
The reverse contrast (faces looking away from each other vs. faces looking at each other) activated the fronto-parietal attention network, including left precentral gyrus, right middle frontal gyrus, and right superior parietal lobe. As participants were exposed to two gaze trajectories when viewing faces that were looking away from each other as opposed to looking at each other, it is probable that more shifts of spatial attention were elicited in the no attentional relation condition. As pointed out earlier, the question whether overt eye movements were elicited in addition to covert shifts of attention needs to be addressed in future studies. In addition, processing faces looking away from each other yielded activation in widespread areas around lingual gyrus, bilateral insula, right anterior cingulate cortex, left orbito-frontal gyrus, left middle temporal gyrus and temporal pole, left precuneus, left angular gyrus, and right thalamus. Many of these areas have been assigned functions in social cognition and visuospatial attention (precuneus, angular gyrus: Cavanna & Trimble, 2006;Hillis et al., 2005;Schurz, Radua, Aichhorn, Richlan, & Perner, 2014), in face processing (lingual gyrus: Batty and Taylor, 2003;Clark, Maisog, & Haxby, 1998;Martinez et al., 1999), in socioaffective processing and directing attention to salient events (insula, anterior cingulate cortex: Menon & Uddin, 2010;Singer et al., 2004), and in (emotional) action understanding as well as theory of mind (middle temporal gyrus, temporal pole: Nummenmaa & Calder, 2009;Olson, Plotzker, & Ezzyat, 2007;Saxe, 2006;Schultz et al., 2004;Schurz et al., 2014). Taken together, faces looking away from each other seem to elicit (1) attentional and visuo-spatial processes, that are due to the fact that faces looking away from each other provide more complex spatial information, and (2) affective and cognitive processes involved in social attention and cognition that may be related to the attempt to understand the implication of two faces looking away from each other. Hence, in the context of either observing the presence or the absence of an attentional relation, processing of two faces looking away from each other seems to be accompanied by enhanced requirements on spatial attention and social cognition.

Observed attentional relation and gaze congruency
The main question of the present study concerned the neural basis of the effect of observed eye contact on subsequent gaze following. In line with previous studies (Böckler et al., 2011(Böckler et al., , 2014, a behavioral gaze-congruency effect was found after observed eye contact, but reduced after faces had looked away from each other. What processes underlie this interaction? If observed eye contact leads to a processing benefit for congruent gaze cues, we expected to find enhanced activation in frontoparietal and temporal areas for congruent trials after observed eye contact (vs. averted gaze). If, by contrast, the interaction was due to reduced processing costs for incongruent trials when faces had been looking away from each other, we should have found enhanced activation for incongruent trials when faces had looked at (vs. away from) each other.
Our neuroimaging data suggest that processing of congruent gaze cues was facilitated when provided by gazers who had previously looked at each other as compared to away from each other. Comparing congruent trials in the eye contact with the averted gaze condition revealed higher activation in frontoparietal areas (right precentral sulcus and left inferior parietal lobe), suggesting that spatial attention to those gaze cues was enhanced after observing eye contact between the gazers. In addition, an increase in activation was found in left middle temporal gyrus, an area that has previously been reported to be particularly active when faces provided congruent gaze cues after establishing eye contact with the viewer (Bristow et al., 2007). This area seems to play a role in social attention and in the understanding of intentional actions and communicative signals such as gestures (Xu et al., 2009). Hence, besides the enhancement of activation in areas typically reported for (gaze-induced) attention shifts (e.g., Pelphrey et al., 2003;Schilbach et al., 2010), observed eye contact previous to gaze cues induced increased activation in areas involved in processing of communicative signals. This finding suggests that observed eye contact, similar to being the recipient of direct gaze (Bristow et al., 2007;Senju & Csibra, 2008), may serve as an ostensive signal that indicates the relevance of subsequent gaze cues (Csibra & Gergely, 2009). When these gazes were then directed toward the target (congruent), an intentional relation between gazers and object and/or a communicative signal from the gazers to the participant may have been inferred. Importantly, activation elicited in incongruent trials was not modulated by the attentional relation of the gazers, a finding that mirrors behavioral results. Hence, in contrast to congruent gaze target relations, the processing of incongruent gaze cues was not modulated by the attentional relations between the gazers. Incongruent trials pose higher visuo-spatial processing demands than congruent trials and may thereby be generally more immune to contextual influences. Alternatively (or additionally), ostensive signals may be particularly influential in the processing of congruent trials because these trials clearly depict a (perceptual or intentional) relation between the gazers and the target.
Comparing incongruent gaze cues and congruent gaze cues in the no attentional relation condition revealed enhanced activation in fronto-parietal attention areas and right middle occipital gyrus, areas that have all been reported in processing incongruent gaze cues (Bristow et al., 2007;Dichter & Belger, 2007;Itier & Batty, 2009;Pelphrey et al., 2003Pelphrey et al., , 2004Schilbach et al., 2010). Interestingly, this fronto-parietal activation pattern was very similar to the pattern revealed for the contrast of congruent gaze cues in the attention relation condition versus congruent gaze cues in the no attentional relation condition, a finding that further points toward a decrease in gaze processing when faces that have not established eye contact jointly look at a target (i.e., providing congruent gaze cues).

CONCLUSION
The main aim of the present study was to investigate the processes underlying the modulation of gaze following by observing eye contact (Böckler et al., 2011). Behavioral results revealed an enhanced gazecueing effect when the gazers had previously established eye contact. Neuroimaging results indicate that this effect is due to a processing benefit for congruent gaze cues after observing eye contact (as opposed to observing averted gaze). Our findings suggest that not only when eye contact is established with the viewer (Bristow et al., 2007;Senju & Csibra, 2008) but also when eye contact is merely observed between others can it function as a social signal that enhances processing of the others' subsequent actions (e.g., gaze cues) (Csibra & Gergely, 2009). Hence, in interactions with other people, we do not only pay attention to their gaze when it is framed as being directly relevant to us (i.e., when they have addressed or looked at us before), but also when it takes place in the context of an (attentional) interaction that we are not part of.
Like gaze following (i.e., sensitivity to the attentional relation between another person and an object or the environment) and the direct gaze effect (i.e., sensitivity to being engaged in an attentional relation with another person), enhanced processing of others' gaze after observing an attentional relation between them could play a crucial role in learning (Csibra & Gergely, 2009), especially in observational learning (Bandura et al., 1966). In addition, immediately spotting eye contact between others and adjusting one's attentional mind-set accordingly may be beneficial when we have to temporally and/or spatially coordinate our actions with those of others, like when building something or playing music together (Clark & Krych, 2004;Williamson & Davidson, 2002). Finally, sensitivity to the relation between others can be crucial for social behavior. Humans are extremely susceptible to cues that signal social exclusion from an interaction (Williams, 2007) or an attentional relation (Böckler et al., 2014). The sensitivity to (potentially) being excluded may, in turn, help to enhance behavioral tendencies that re-establish social bonds (e.g., mimicry; Lakin et al., 2008). Hence, our sensitivity to attentional relations between others may be a crucial building block for joint actions that involve more than two actors and for organizing our social life in general.

Original manuscript received 19 January 2015
Revised manuscript received 16 March 2015 First published online 1 July 2015