Eyes or mouth? Exploring eye gaze patterns and their relation with early stress perception in European Portuguese

Previous research has shown that eye gaze patterns relate to language development, with more attention to the mouth signaling ongoing acquisition. We examined infants’ eye gaze in a stress perception experiment, in which European Portuguese (EP) learning infants showed a preference for the iambic stress pattern. Specifically, we asked whether there was a relation between eye gaze patterns and the preferred stress pattern. Eye gaze patterns of 25 monolingual typically developing infants aged 5–6 months old were examined using eye-tracking. Our results show that, although an interaction between looks to the area of interest (face, eyes, mouth, and arm) and stress preference was not found, eye gaze to the mouth region (and to the face) was modulated by the stress pattern, with more attention to the mouth in infants that do not show an iambic preference. These findings add further support for infants’ use of eye gaze in early language development. They also highlight the need for multimodal approaches for a better understanding of language development. In the particular case of the challenging topic of the acquisition of stress in European Portuguese, they provide converging evidence for an advantage of iambic stress in early development. (195 words).

Some studies show that language (un)familiarity and age also play a crucial role in the use of visual cues to language discrimination. For instance, Weikum, Vouloumanos, speech perception. It has been reported that motor areas get particularly involved during the perception of non-native speech compared to native speech, when the later has already been acquired (Kuhl, Ramírez, Bosseler, Lin & Imada, 2014). On the other hand, more attention to the mouth could indicate instead a more advanced learning stage, as some studies found a relationship between increased attention to the mouth and both concurrent and predictive expressive language skills (Tenenbaum et al., 2015;Tsang et al., 2018).
In the current study, we examine infants' eye gaze patterns in a stress perception experiment, in which European Portuguese (EP) learning infants showed a clear preference for the iambic stress pattern (Frota, Butler, Uysal, Severino & Vigário, submitted). Specifically, we asked whether there was a relation between eye gaze patterns and the preferred stress pattern. This is the first study on EP-learning infants' gaze patterns to talking faces, and, to the best of our knowledge, on the relation between infants' eye gaze to communicative faces and the acquisition of lexical stress. Assuming that the preferred stress pattern is acquired earlier (Bhatara, Boll-Avetisyan, Hohle, & Nazzi, 2018;Jusczyk, Cutler & Redanz, 1993), an iambic preference indicates a more advanced learning stage. If increased attention to the mouth signals ongoing acquisition, we predict more attention to the mouth in EP-learning infants that do not show an iambic preference, and less attention to the mouth in infants showing the dominant iambic preference. If, by contrast, more attention to the mouth is related to more advanced language abilities, the reverse pattern is predicted. The current study will thus contribute to deepen the current understanding of the relation between eye gaze patterns and language development.

Participants
Twenty-seven monolingual typically developing infants participated in this study (15 males; mean age: 5 months 14 days; age range: 5 months 2 days to 6 months 27 days), recruited from the wider Lisbon area. Five-six month olds were chosen for two main reasons. First, stress discrimination or a processing advantage for a given stress pattern has been reported already at such an early age, at least in languages like German and French (Friederici, Friedrich & Christophe, 2007;Weber, Hahne, Friedrich & Friederici, 2004), and the preference for a given stress pattern has been suggested to emerge between 4 and 6 months (Höhle, Bijeljac-Babic, Herold, Weissenborn & Nazzi., 2009, for German). Second, our age group is comparable to that of infants in other studies exploring the role of visual attention in speech perception, before the developmental shift towards the mouth.
Typical development was assessed after the experimental task, within a follow-up procedure that was part of the EBELa Project (http://labfon.letras.ulisboa.pt/babylab/ EBELa/). Two screening tools were used: (i) the Communication and Symbolic Behavior Scales Developmental Profile (CSBS-DP) Checklist (Wetherby & Prizant, 2003) adapted for EP (Frota, Vicente, Filipe, & Vigário, 2014-2016, and filled in by all caregivers when infants were between 6 and 24 months of age; and (ii) The Portuguese MacArthur-Bates Communicative Development Inventories (CDI) short forms (Frota, Butler, Correia, Severino, Vicente, & Vigário, 2016), filled in by caregivers when infants were between 8 and 30 months. According to these developmental tools, all children exhibited social communication, language and symbolic functioning skills as expected for their age (including eye gaze, gestures, productive and receptive vocabularies).
Two infants were excluded from the analysis of eye gaze patterns due to poor tracking ratio (see section 2.2.1 for detailed information on exclusion criteria). From the 25 infants included in the gaze pattern analysis, three were excluded from the stress perception analysis due to poor tracking ratio in that task (Frota et al., submitted). A final sample of 22 infants was thus used for the analysis of eye gaze patterns in relation to stress perception. relation with early stress perception in European Portuguese Art. 4, page 4 of 13

Materials and procedure
Both stress perception and eye gaze data to talking faces were obtained from a perception experiment run in a sound attenuated room. Infants were sat in an appropriate chair or on their parent's lap, at a distance of approximately 70 cm from the Dell LCD screen (1680 × 1050 pixel resolution) of a remote eye-tracker (SMI RED500). Stimuli presentation and data storage were performed with the SMI Experimenter Center and iView X software.

Eye gaze pattern to talking faces
The analysis of the gaze pattern was based on a 4-second long dynamic video of a talking movie character (Noddy), presented at the end of each block of the perception experiment to keep infants engaged in the task. The video was carefully chosen to allow measurement of eye gaze to talking faces and social gestures. The talking character moves its eyes, mouth, head, and arm while talking, against a colorful scenery (Figure 1). The choice of the video was also motivated by the participation of atypically developing populations, such as infants at risk for autism, given that the perception experiment was part of larger projects on typical and atypical development (EBELa, http://labfon.letras.ulisboa.pt/ babylab/EBELa/; Horizon21, http://labfon.letras.ulisboa.pt/babylab/horizon21/; PLOs, http://labfon.letras.ulisboa.pt/babylab/PLOS/en/). Some atypical populations are known to prefer cartoon-faces over real-faces (Rosset et al., 2007;Van Der Geest, Kemner, Verbaten & Van Engeland, 2002; inter alia), and thus a talking movie character would allow comparing findings across the different groups of children.
Four different exemplars of the video were created, each containing a different encouraging speech message while the visual features were kept the same. The four exemplars were carefully designed so that audio and video were synchronized, namely, the speech messages were carefully constructed to match the video movements (mouth, head, arm) and were then synchronized with the video. Speech messages were produced in child directed speech by a female native speaker of EP. Order of presentation of the videos was fixed within participants and randomized across participants.
For each video, three dynamic Areas of Interest (AOIs) -face, eyes, mouth -and one nonhead dynamic AOI -waving arm -were defined, as illustrated in the total screen, the face covers 8,2% of the screen, the eyes cover 2,4%, the mouth covers 0,5%, and the arm covers 2,3%. Eye gaze to the background was not included in this analysis; only visual communicative cues (face and arm) were explored. A sample video is provided as an Additional file.
A total of 101 trials were initially considered for analysis (mean 3,74 by infant). After inspection of the participants' tracking ratio for each video trial, all trials with a tracking ratio below 50% were discarded. Thus, 34 videos were excluded (33,66% of the initial set). This resulted in the exclusion of two infants, for whom there was not enough data for analysis. The excluded videos had a mean tracking ratio of 9,55% (SD = 29,90%). The included videos had a mean tracking ratio of 89,04% (SD = 39,89%).
Net dwell time (in milliseconds) for each AOI was used as the eye gaze measure.

Stress perception
Infant stress perception was examined using a modified version of the Anticipatory Eye Movement paradigm. Infants were exposed to trials with two geometric images, each associated to a stress pattern (iambic or trochaic). In the test phase, both stress patterns were presented and infants were expected to look to the side of the screen where the image associated to the pattern they were listening to appeared in the training phase.
Training and test phase constituted a block, and the experiment included 8 blocks. Each block ended with an exemplar of the Noddy video described above. Infants performed a minimum of two and a maximum of six blocks (with a mean of 4 blocks), depending on their interest in the task. Disyllabic segmentally varied nonsense words with either penult or final stress were used. Speech stimuli were uttered by a female native speaker of EP in child directed speech style (cf. Frota et al., submitted).

Results and discussion
Not all infants watched all video exemplars. Due to the unequal sample size per encouraging message across infants, a Wilcoxon test was run to compare net dwell times (ms) between each pair of video exemplars. The different encouraging messages were shown not to impact on the results, as overall eye gaze to the different videos was not significantly different

Eye gaze
The eye gaze patterns found are shown in Figure 2. Eye gaze was concentrated more on the face than on the arm (t(24) = 6.836, p < .001, d = 1.37; mean face: 2149 ms, mean arm: 284 ms). In addition, infants overall attended more to the eyes than to the mouth (t(24) = 4.219, p < .001, d = .84; mean eyes: 1026 ms, mean mouth: 241 ms). Thus, the eye gaze of EP-learning infants at 5-6 months of age shows a similar pattern as reported in other studies, with a general preference for the eye region at this young age (Ayneto & Sebastián-Gallés, 2017;Fort et al., 2018;Lewkowicz & Hansen-Tift, 2012;Pons, Bosch & Lewkowicz, 2015). Frota et al. (submitted) demonstrated that EP learning infants show a preference for the iambic stress pattern, as they attended longer to the iambic side than the trochaic side of the screen during the test phase, irrespective of being exposed to the iambic or trochaic stress pattern.

Eye gaze pattern and stress preference
Considering that the preferred stress pattern has been reported to be acquired earlier (Jusczyk, Cutler & Redanz, 1993, among others), an iambic preference indicates a more advanced learning stage. If increased attention to the mouth is related to ongoing acquisition, more attention to the mouth is expected in EP-learning infants that do not show an iambic preference, and less attention to the mouth in infants showing the dominant iambic preference. If, by contrast, more attention to the mouth is related to more advanced language abilities, the reverse pattern is predicted.
To explore the relation between eye gaze patterns and stress preference, we used the net dwell time data by participant from Frota et al.'s (submitted) study. The data was divided into the following two groups: infants who looked longer to the iambic side constituted the 'iamb' group (15 infants); infants who looked longer to the trochaic side, and infants who showed a balanced looking to either side of the screen formed the 'non-iamb' group (7 infants). A difference in looking time of more than 90 ms was taken as an indication of preference for a given stress pattern. This threshold was obtained by considering the mean difference in looking time to the two stress patterns (376 ms) minus the standard deviation (266 ms), adjusted to the closest difference in looking time to the two stress patterns (93 ms).
A mixed ANOVA was run with AOI as a within-subjects factor (4 levels: face, eyes, mouth, arm), and stress preference as the between-subjects factor (2 levels: iamb, and non-iamb). A main effect of the AOI was observed (F(3,60) = 47.368, p < .001, η = .84). There was no effect of stress preference (F(1,20) = 2.113, p > .05, η = .32) and no interaction between AOI and stress preference (F(3,60) = 1.589, p > .05, η = .26). A closer look at the data shows a strong dominance of the face and eyes overall, which underlies this result (Figure 3). However, beyond the overall result, eye gaze to the mouth region (red bar) -and face (orange bar) -was found to be modulated by stress preference, as there is more attention to the mouth in infants that do not show an iambic preference (Figure 3). To ascertain the strength of this effect, each AOI was analyzed separately with stress preference as a between-subjects factor. A significant effect was found for mouth Furthermore, a moderate correlation was found between stress preference (iamb, noniamb) and eye gaze to the mouth region (r = .574, p < .01). A more detailed analysis of the correlation pattern showed that looks to the mouth were positively correlated with looking time results for the non-iambic pattern (r = .478, p < .05), as shown in Figure 4. Conversely, eye gaze to the eye region was positively correlated with looking time results for the iambic pattern (r = .469, p < .05), as illustrated in Figure 5. No other significant correlations were found.  These results show that infants' eye gaze to talking faces is related to their stress preference in a perception task. EP-learning infants showing the dominant iambic preference attend less to the mouth; by contrast, infants that do not display a preference for iambic stress attend more to the mouth. Given that the preferred stress pattern is the first to be acquired, and its acquisition has been reported to develop between 4 and 6 months (depending on the language), this suggests that increased attention to the mouth is recruited to support ongoing language acquisition, in this particular case the development of lexical stress perception. Further support for the link between a more advanced learning stage, signaled by the iambic preference, and less attention to the mouth is provided by later language measures taken from the follow up study mentioned in section 2.1. Looks to the mouth at 5-6 months were found to be negatively correlated with receptive vocabulary measured at 12-18 months using the Portuguese CDI short forms (r = -.60, p < .05). In other words, the present findings show that less attention to the mouth at 5-6 months relates to more advanced language abilities.

Discussion
Eye gaze patterns to talking faces have been shown to relate to communicative and social issues, as well as to language development. However, previous studies have suggested different understandings of the relation between eye gaze patterns and language development. On the one hand, more attention to the mouth seems to signal a less advanced learning stage, or ongoing acquisition, as shown by its relation to emerging articulatory abilities and/or, increased processing effort (Ayneto & Sebastián-Gallés, 2017;Fort et al., 2018;Hillairet de Boisferon et al., 2017;Lewkowicz & Hansen-Tift, 2012;among others). On the other hand, more attention to the mouth seems to indicate a more advanced learning stage, given the relation between increased attention to the mouth and both concurrent and predictive expressive language skills (Tenenbaum et al., 2015;Tsang et al., 2018).
In the present study, we investigated European Portuguese learning infants' eye gaze patterns to talking faces. Specifically, we examined 5-6 month old infants' eye gaze using a communicative video included in a stress perception experiment, in which infants showed a preference for the iambic stress pattern (Frota et al., submitted). We asked whether there was a relation between eye gaze patterns to different areas of interest in the communicative video (face, eyes, mouth, arm) and the preferred stress pattern. To the best of our knowledge, no study had previously addressed the relation between infants' eye gaze to communicative faces and the acquisition of lexical stress. Considering that the preferred stress pattern has been reported to be acquired earlier, an iambic preference indicates a more advanced learning stage. If increased attention to the mouth is indeed related to ongoing acquisition, or processing effort, more attention to the mouth was expected in EP-learning infants that do not show an iambic preference, and less attention to the mouth in infants showing the dominant iambic preference. If, by contrast, more attention to the mouth is related to more advanced language abilities, the reverse pattern was predicted. The analysis of eye gaze patterns showed that typically developing EP-learning infants aged 5-6 months old looked more to the face than to the arm, and that their attention was focused on the eyes over the mouth. This is in line with studies for other languages, showing the same attentional bias for faces, already shown by newborns (Di Giorgio et al., 2012;Gliga et al., 2009;Johnson et al., 1991), and points to a similar developmental path as that of English-, Catalan-or Spanish-learning infants of the same age (Lewkowicz & Hansen-Tift, 2012;Pons, Bosch & Lewkowicz, 2015). Since this attentional bias is informative of infants' social and language development, in further research we are examining whether eye gaze to talking faces in other groups of infants, including clinical populations and at risk infants, can be an early marker of (a)typical language development (Pejovic, Cruz, Severino & Frota, in progress), as it has been suggested (Annaz et al., 2009;Jones & Klin 2013;Åsberg Johnels et al., 2014;Irwin & Brancazio, 2014;inter alia).
Besides the overall dominance of eye gaze to the face and the eyes, infants' gaze to the mouth region (and face) was found to be modulated by the stress pattern. More specifically, infants who did not show the general preference for iambic stress tended to look more to the mouth than infants who demonstrated a preference for iambic stress. The general preference for iambic stress in 5-6 month old EP-learning infants found in Frota et al.'s (submitted) is in line with recent behavioral and neurophysiological findings for EP adult speakers (Lu, Vigário, Correia, Jerónimo & Frota, 2018). Lu and colleagues reported a processing advantage for the iambic stress pattern over the trochaic pattern manifested in more accurate and fast responses in behavioral tasks, as well as in an increased MMN in the ERP task. Taken together, the results of both infants and adults indicate that iambic stress is the perceptually more salient pattern, and thus the pattern that is easier to process.
In short, the present eye gaze findings confirm the prediction that increased attention to the mouth is recruited to support ongoing language acquisition and processing effort evidenced by EP-learning infants who do not show an iambic preference. By contrast, less attention to the mouth in infants showing the dominant iambic preference supports the relationship between attending less to the mouth at 5-6 months of age and a more advanced learning stage.
In conclusion, the eye gaze exploration of talking faces by 5-6 month old EP-learning infants adds to previous findings suggesting that increased support from visual cues signals ongoing language acquisition, thus contributing to the current understanding of the relation between eye gaze patterns and language development. Additionally, the present study brings novel data relating eye gaze exploration of talking faces and early stress perception, that provide further support for infants' use of eye gaze in early language development. The current findings also highlight the need for multimodal approaches for a more comprehensive study of language development. In the particular case of the challenging topic of the acquisition of stress in European Portuguese, they provide converging evidence for an advantage of iambic stress in early development. relation with early stress perception in European Portuguese Art. 4, page 10 of 13

Additional File
The additional file for this article can be found as follows: • Sample Video. Dynamic AOIs. DOI: https://doi.org/10.5334/jpl.240.s1