An ERP study of hemispheric differences in perceptual representations of language reveals meaning attribution in the right hemisphere and constituents of the N400-effect

Recent findings have revealed that the right hemisphere (RH) is uniquely involved in integrating perceptual information from linguistic input to simulate a mental model of that input. We extend on these findings by testing whether meaning is generated from such models. Participants (N = 37) heard auditory passages describing the visuospatial arrangement of elements into a perceptual representation of a familiar object, then judged whether a laterally-presented target word matched the object. We found a central N400-effect for left visual-field targets, suggesting that meaning was also accessible to the RH. There was no statistical difference for right visual-field targets. Principle component analysis of the data revealed that the N400-effect was driven by positive components. Consequently, the results suggest that i) RH contributions to language comprehension include integrative and perceptual processes that enable overall meaning to be generated from representations of discourse, and ii) positive ERP components may produce N400-effects.


Introduction
Language comprehension involves multiple processes. These processes recruit diffuse brain regions, some of which are lateralised to one cerebral hemisphere. Lateralised language processing is often exemplified by greater left hemisphere (LH) activity in the perisylvian regions during word, sentence, and discourse processing (Just et al., 1996;Vigneau et al., 2006). However, increased activity in areas of the right hemisphere (RH) that are homologous to LH language areas has also been reported during sentence processing (Just et al., 1996). Unique patterns of RH activation are also reported during narrative comprehension (Vigneau et al., 2011), suggesting that RH processes are indeed distinct from LH processes.
Evidence also suggests that hemispheric contributions can reflect stages of discourse comprehension. For example, Xu et al. (2005) found that activity was predominantly localised to the LH at the onset of narrative processing; however, RH activity increased during the resolution of the story, suggesting that the LH was more involved in propositional processing of factual, descriptive information, whereas the RH was more involved in generating a cohesive representation of the overall narrative and extracting a theme or meaning. This is consistent with earlier findings by Nichelli et al. (1995) who found greater activity in a subset of RH regions, relative to the LH, when interpreting the overall theme of a story. Xu et al. (2005) also found greater RH activity in cases where contextual complexity was increased. Potentially, the difficulty with which novel input is integrated into an existing model predicts the extent of RH involvement and therefore activity. Even within the RH, task-specific comprehension subprocesses may predict activity in different RH-networks (Ferstl, Neumann, Bogler, & von Cramon, 2008).
These findings represent a small part of the body of evidence showing that the RH occupies an important role in higher-level language processing and that it does not merely support LH specialised language networks, but rather uniquely contributes to discourse comprehension. However, one unresolved question regards the specific nature of this RH contribution.
One possibility is that the RH's superiority in generating and maintaining mental models of discourse reflects a primary RH contribution to mental imagery. That is, increased RH involvement is consistently observed during semantic judgment tasks that necessitate mental imagery (e.g., Dhond, Witzel, Dale, & Halgren, 2007;Huang & Federmeier, 2015).
From this perspective, discourse comprehension deficits in patients with RH damage (RHD), including deficits in integrating information across sentences, producing a coherent representation of discourse, and generating inferences from discourse representations (Johns, Tooley, & Traxler, 2008;Kahlaoui, Scherer, & Joanette, 2008;Mitchell & Crow, 2005), could be explained by an inability to effectively construct mental models of linguistic input (McDonald, 2000). For example, Marini, Carlomagno, Caltagirone, and Nocentini (2005) found that the ability to generate a cohesive narrative from unordered cartoon picture stories was significantly impaired in patients with RHD. Lincoln et al. (2008) also found that match effects relying on the correct perceptual representation of an object (e.g., a sliced apple vs. a whole apple) were impaired in RHD patients. The nature of these impairments suggest that the RH occupies a special role in accessing perceptual information, integrating perceptual information from linguistic input, and deriving meaning from integrated perceptual representations.
Accordingly, we propose that neural networks within the RH contribute to language comprehension by integrating perceptual information into a 'situation model'-a mental simulation of text information thought to be the foundation of discourse comprehension (Zwaan & Radvansky, 1998). Male and Gouldthorp (2020a) tested this proposal in an ERP study in which participants listened to auditory prime passages-each comprising several sentences that described the construction of a stylised image of a familiar object. For example, "Draw a long teardrop shape. Colour the teardrop in green. Now, draw a short brown vertical line extending down from the base of the teardrop." Thereafter, participants indicated whether a lateralized target image matched the mental image they had created. Target images were either a match, depicting the object with individual perceptual elements integrated into a coherent image as per prime passages (e.g., a leaf); an unintegrated mismatch, displaying individual elements described in the prime passages (e.g., displaced teardrop shape with a disconnected brown line in a separate location) but not spatially integrated into a coherent whole; or an unrelated mismatch (e.g., a moon) (see Male & Gouldthorp, 2020b, for further stimulus details). Male and Gouldthorp (2020a) found that for RVF-targets, N300 amplitudes in the LH did not differ for the integrated match and unintegrated mismatch targets, suggesting that the LH represented individual perceptual elements, but did not spatially integrate them.
For LVF-targets, N300 amplitudes differed significantly for integrated match and unintegrated mismatch targets, indicating inclusion of visuospatial information in perceptual representations. However, a critical unanswered question is whether generation of a mental representation in the RH also results in attribution of meaning (i.e., contains a linguistic component) or whether it is purely a visuospatial representation (i.e., without a linguistic component). That is, whether the semantic label of an integrated mental representation of a priming passage is accessible. This would demonstrate a RH contribution to language comprehension of not only constructing a visuospatial representation from linguistic inputs, but also in generating meaning from this mental image.

The present study
The present study was intended to delineate the RH's role in accessing meaning from a mental model of an object, further elucidating the nature of the RH contribution to discourse processing. We used a similar paradigm to the one above, but replaced lateralised target images with lateralised target words, specifically, the word label for the integrated or unrelated image (e. g., leaf or moon). Therefore, correct responses to the semantic label of the image necessitates not only the perceptual representation and integration of each of the perceptual elements described, but also the attribution of meaning to this combination of elements, thus allowing comparison of each hemisphere's attribution of meaning to representations based on perceptual information extracted from language (see supplementary materials for a complete list of stimuli).
As N300 amplitudes index the semantic processing of images (McPherson & Holcomb, 1999) and N400 amplitudes are inversely related to the degree of semantic relatedness of word targets to prime stimuli, we anticipated larger N400 amplitudes for mismatch vs. match word targets. Evidence of semantic facilitation vis-à-vis the N400 congruency effect-less negativity for congruent than incongruent word targets-has been used to reveal differences in hemispheric processing in divided visual-field paradigms (e.g., Federmeier & Kutas, 1999;Wlotko & Federmeier, 2013). Accordingly, if the RH attributes meaning to the integrated image, this will facilitate match target processing and we will observe larger N400 amplitudes for mismatch vs. match LVFtargets (i.e., the N400-effect). Alternatively, if the RH represents spatially integrated perceptual information without attributing meaning to it, N400 amplitudes will not differ for mismatch vs. match LVFtargets. If the LH does not perceptually integrate linguistic input, it will have no representational basis to attribute meaning to, and N400 amplitudes will not differ for mismatch vs. match RVF-targets.

Participants
Thirty-seven 1 (mean age 24, 18-50 years) self-declared neurologically healthy, native English-speaking adults (7 males), with normal or corrected-to-normal vision, and normal hearing, volunteered their time in exchange for either course credit or a chance to win an AUD$50 shopping voucher. Participants were primarily right-handed, revealed in a mean handedness quotient of 0.8 (Bryden, 1982). No participant had participated in the previous study (Male & Gouldthorp, 2020a). All participants provided informed written consent. The Murdoch University Ethics Committee approved the experiment (2018/068).

Experiment
As the perceptual elaboration paradigm is thoroughly detailed elsewhere (see Male & Gouldthorp, 2020a;2020b), we briefly outline the differences between it and the current study. We used a 2 × 2 repeatedmeasures design, with visual-field (VF: RVF vs. LVF) and target condition (TC: mismatch vs. match) as factors.
Stimuli included 80 passages, each comprising at least three sentences describing the integration of perceptual components into a stylised image of a common object. Headphones delivered auditory passages binaurally while participants focused on a white fixation cross on the grey background of a CRT monitor (16-inch display, 800 × 600 resolution, and 85 Hz refresh rate). Printed black word targets were all concrete nouns, between 3 and 6 letters long, presented in upper case, size 42, Courier font. Targets either matched or mismatched the image depicted by the auditory passage. Stimuli were presented in 20 blocks of four trials, randomly drawn from each combination of conditions (i.e., LVF-mismatch, LVF-match, RVF-mismatch, and RVF-match)-totalling 20 trials per condition. Targets appeared for 150 ms 2 and the time between the end of the auditory passage and the target word was always 1500 ms. 3

EEG recording and data analysis
EEG recording, pre-processing, and analyses were similar to that 1 Participants with <60% (hit rate) accuracy overall or <11 correct trials (55% hit rate) in any condition were excluded. As a result, we do not present the data from a further 12 participants in any section of the current study.
2 A closed-circuit television (CCTV) was positioned above the monitor to confirm participants maintained central fixation during trials. 3 To ensure timing precision, a photocell was positioned on the monitor screen as an additional timing measure of target onset and offset.
In the supplementary analysis, epochs were time-locked to the offset of the auditory prime. Epochs began 1600-ms prior to target onset and ended at target onset. As in León-Cabrera, Rodríguez-Fornells, and Morís (2017), the 100 ms pre-delay period was used as baseline. We compared correct and incorrect responses and retained 48 (±10, 28-68) correctresponse epochs and 12 (±5, 3-20) incorrect-response epochs. Fig. 1 shows electrodes within the eleven regions of interest (ROIs). These were the left (L) and right (R) frontal (F), temporal (T), central (C), and parieto-occipital (PO) regions. We also defined midline (M), F, C, and PO regions. All electrodes were included in the temporal principal component analyses (PCAs).
For Bayesian analyses of variances (ANOVAs) and t-tests, we considered a Bayes factor (BF) close to 1 as weak or insensitive (Dienes, 2014). Where appropriate, we corrected p values using the Bonferroni correction. All remaining details replicate Male and Gouldthorp (2020a). Table 1 shows the mean accuracy (dʹ) and reaction time (RT) 4 for target conditions in the LVF and RVF. There was a significant weak positive correlation between accuracy (dʹ) and RT, r(1 4 6) = 0.192, p = .019 (two-tailed), with faster responses correlated with lower accuracy. This is indicative of speed-accuracy trade-off effect.

Accuracy
For accuracy, the 2 × 2 repeated-measures (Bayesian and Frequentist) ANOVAs, with VF and TC as factors, revealed a significant VF × TC interaction, F(1, 36) = 4.774, p = .035, η 2 = 0.014. The interaction emerged due to lower accuracy for match compared to mismatch targets in the LVF, but not in the RVF (Table 1). There was no difference between mismatch targets, t(36) = -0.386, p = .702, BF 10 = 0.190. Correspondingly, accuracy was significantly lower for LVF-match compared to RVF-match targets, t(36) = 2.261, p = .030, BF 10 = 1.675. The data provide strong evidence for the favoured model including only the main effect of TC (BF 10 = 63.900) and weak evidence for including the moderating effect of VF on TC (BF Incl = 1.390).

RT
For RT, we found a main effect of TC only, F(1, 36) = 48.960, p < .001, η 2 = 0.081. Table 1 shows that RT was faster for match targets compared to mismatch targets, suggesting that prime passages facilitated faster responses to congruent targets. The data provide inconclusive evidence for including the effect of VF (BF Incl = 1.148) and the difference between the model including both main effects and the model including TC alone is marginal (BF 10 = 1.273e+9 vs. BF 10 = 9.197e+8).

Event-related potentials (ERPs)
In Fig. 1, we show (a) ERPs in each ROI for targets appearing in the RVF (left panel) and LVF (right panel), and (b) ERPs in each ROI during the anticipatory period prior to target onset. In both cases, the data was reduced to principal components explaining 95% of variance according to the corresponding PCA.

Anticipatory ERPs
We explored the electrophysiological activity during the 1500-ms delay between the prime passage offset and target onset for any evidence of anticipation. We found increased negativity beginning approximately 400 and 700 ms after prime offset at C and PO ROIs, respectively. Based on recent work (León-Cabrera et al., 2017), we compared amplitudes in the final 200 ms of the delay-period from all ROIs for the pre-target ERPs.
A 11 × 2 repeated-measures ANOVA with ROI and response accuracy (RA) as factors revealed a main effect of ROI only, F(1, 10) = 3.665, p = The distribution of this slow negative wave potential resembles stimulus preceding negativity (SPN, Brunia, 1988;Brunia & Damen, 1988;Damen & Brunia, 1987). SPN can occur in anticipation of sensory input and motor response preparation (Brunia & Damen, 1988)-both of which occur in the current study. Undifferentiated SPN is also RH dominant (Damen & Brunia, 1987). We consider the RH dominant SPN as further evidence that participants were indeed attending to task.

Principle component analyses (PCAs)
PCAs revealed seven principal components for LVF-targets and eight principal components for RVF-targets. To determine which components were responsible for the N400-effect, we conducted 6 (ROI: L-C, M-C, R-C, L-PO, M-PO, R-PO) × 2 (TC: mismatch vs. match) repeated-measures ANOVAs, on component scores at the C and PO ROIs-where we found significant mismatch vs. match ERP differences (s. 3.2.2)-for components whose peak latency was at least 250 ms after target onset. Fig. 2 shows those components for which analysis revealed a significant main effect, or interaction with, TC.
In the RVF condition, we found two components whose scores differed for mismatch vs. match targets. 5 Component 1 peak latency was 540 ms. Component maximum (i.e., positive pole) was widely distributed over centroparietal regions such that ROI did not modulate its activity, F(5, 180) = 2.488, p = .083, ε = 0.449, η 2 = 0.027, or the significant TC main effect, F(1, 36) = 6.521, p = .017, η 2 = 0.015, ROI × TC: F(5, 180) = 1.713, p = .182, ε = 0.465, η 2 = 0.004, of greater positivity for match compared to mismatch targets. Correspondingly, we emphasize at all six ROIs for component 1 in Fig. 2. The data provide very strong evidence for the model including the main effect of ROI and TC (BF 10 = 86.828). The data also provide strong evidence against Fig. 1. Grand average ERPs for target words appearing in right visual-field (RVF; upper leftmost array) and left visual-field (LVF; upper rightmost array)-activity is time locked to target word onset-and for pre-stimulus interval (lower array)-activity is time locked to the auditory prime offset. In each array, we show ERPs from the left (L) and right (R) frontal (F), central (C), temporal (T), and parieto-occipital (PO) regions. We also show ERPs from the midline (M) F, C, and PO regions. Electrode clusters within each ROI are depicted by the black circles on the diagram of the Electrical Geodesic dense-array 129-electrode net. We highlight the N1 ERP component for LVF and RVF targets in the contralateral hemisphere-where they are largest. including the ROI × TC interaction (BF Incl = 0.082). Component 5 was largest at 352 ms. Maximum negativity (i.e., negative pole) was posteriorly distributed whereas maximum positivity (i.e., positive pole) was frontally distributed, making it incompatible with the anterior N300 reported by Male and Gouldthorp (2020a). Instead, this component may represent a target P3 (or P3b). The component's magnitude was larger for mismatch than for match targets (i.e., more positive at the positive pole and more negative at the negative pole), thus producing the significant ROI × TC interaction, F(5, 180) = 7.561, p < .001, ε = 0.397, η 2 = 0.019. The data do not provide evidence for the alternative. Instead, the data provide strong evidence against including either main effect (ROI: BF Incl = 0.555, TC: BF Incl = 0.091) or interaction (BF Incl = 0.111) in any Bayesian model. However, separate 3 (ROI) × 2 (TC) ANOVAs at PO and C ROIs revealed a single main effect of TC at PO ROIs, F(1, 36) = 6.285, p = .017, η 2 = 0.024-where mismatch targets evoked greater negativity than match targets-and C ROIs, F(1, 36) = 4.666, p = .038, η 2 = 0.015-where mismatch targets evoked greater positivity than match targets. In both cases, there was no interaction with, or main effect of, ROI. The favoured Bayesian model contained only the main effect of TC and the data provide very strong evidence against including ROI or TC at PO (BF 10 = 40.444, ROI: BF Incl = 0.188, TC: BF Incl = 0.070) and C (BF 10 = 4.523, ROI: BF Incl = 0.087, TC: BF Incl = 0.035) ROIs. As a result, component 5 contributed to the mismatch-minus-match negativity at PO ROIs only.
In the LVF condition, we found one component. The similarity in score comparisons for component 1 and the ERP amplitude analysis for LVF-targets (s. 3.2.2) imply that component 1 is responsible for the mismatch-minus-match negativity in ERP amplitudes at C and PO ROIs. However, at the positive (left-lateral posterior) and negative (right-lateral anterior) poles, component 1 is larger (i.e., more positive or more negative, respectively) for match compared to mismatch targets. Therefore, the mismatch-minus-match negativity reflects greater positivity to match targets in proximity to the positive pole rather than greater negativity to the mismatch target at the negative pole.
Overall, PCAs revealed a late component in the RVF and LVF conditions producing significant mismatch-minus-match negativity at C ROIs. As we did not find a significant difference in amplitudes in the analysis of amplitudes for RVF-targets (s. 3.2.2), it is likely that the already smaller mismatch-minus-match negativity for RVF-targets at C ROIs was reduced even further by the significant mismatch-minusmatch positivity contributed by the second RVF component-component 5. The same late LVF component responsible for the mismatch-minus-match negativity at the C ROIs was responsible for the mismatch-minus-match negativity at the R-PO (s. 3.2.2). Interestingly, no component produced the typical mismatch-minus-match negativity at its site of maximal negativity. Instead, differences reflect greater positivity to match targets at their site of maximal positivity.

Discussion
The present study was conducted to elucidate hemispheric differences in meaning attribution following perceptual elaboration during language processing.
The electrophysiological results revealed early processing differences for targets. We found greater negativity for incongruent mismatch vs. congruent match targets during the canonical N400 time-window. This is consistent with existing congruency research (e.g., Federmeier & Kutas, 1999Stanfield & Zwaan, 2001;Zwaan, Stanfield, & Yaxley, 2002). However, amplitude differences were significant for LVFtargets only, suggesting that perceptual elaboration facilitated semantic processing for targets initially processed by the RH. Specifically, these results suggest that the RH maintained a visuospatially integrated representation and also accessed the corresponding semantic information and that this did not occur for the LH.
Behaviourally, accuracy was better for LVF-mismatch compared to LVF-match targets only, indicating a moderating effect of visual-field on condition, perhaps owing to superior word processing in the LH. We also found faster RTs for match compared to mismatch targets in both visualfields, suggesting that prime passages facilitated faster responses for congruent targets irrespective of initial hemisphere processing. Because a mental representation capable of generating meaning (i.e., a model encompassing the correct visuospatial arrangement) is available to both hemispheres by the time of decision (>660 ms), response latencies are unlikely to reflect earlier processing differences revealed in ERPs. Although RTs did not differ between visual-fields, there was a speedaccuracy trade-off effect indicating a mismatch bias-tendency to favour mismatch responses for targets presented to the LVF in order to maintain a fast response time-producing the significant difference in accuracy between match and mismatch LVF-targets. We suspect that participants accuracy is consistently lower for match targets due to, at least in part, task unfamiliarity. The paradigm was designed to separate processes that are typically entwined during normal discourse comprehension, which-although necessary for scientific experimental design standards-inherently leads to some sacrifice of ecological validity. Although the present study provides insight into these processes as they occur under task conditions that demand perceptual elaboration, an unresolved question is the extent to which these processes occur in naturalistic language comprehension (and under which conditions or circumstances). Nevertheless, other literature has begun to address this question (e.g., Stanfield & Zwaan, 2001;Zwaan et al., 2002).
To reveal those components responsible for our ERP differences, we decomposed the electrophysiological signal into principal components. PCAs did not reveal a traditional N400 component. Instead, we found distinct positive components contributing to the mismatch-minus-match negativity in each visual-field condition. For RVF-targets, we found what might represent a P3b whose amplitudes are larger for mismatch vs. match target words, perhaps due to them being more unexpected than their match counterparts. This may reflect commonalities between word/propositional level representations of the preceding passage and match target word. We also found a late component for LVF-targets and RVF-targets, whose positivity at C and PO ROIs was greater for match than for mismatch targets. These components may represent P600s or constituents of the N400/P600 biphasic complex (Hoeks, Stowe, & Doedens, 2004).
Recent works suggest that the P600-effect-greater positivity to incongruous targets-can index the ease with which semantically-based integration occurs (e.g., Brouwer, Crocker, Venhuizen, & Hoeks, 2017;Delogu, Brouwer, & Crocker, 2019). For example, Delogu et al. (2019) found larger positivity in the P600 time-window for words that were both a) difficult to semantically integrate, and b) related to preceding context-and were therefore plausible. However, there was no effect for words that were unrelated to the preceding context-and were therefore implausible. Khachatryan, Hnazaee, and Van Hulle (2018) also found greater positivity in the P600 time-window for word targets following meaningful sentences-permitting word integration-vs. meaningless sentences-for which attempts at word integration may have been forgone. From this perspective, the greater match positivity could reflect a tendency for integration effort, but only when there is a potential for successful integration (e.g., when the word is not immediately discounted as a mismatch).
Consequently, the congruency effects for LVF-targets we have found could reflect forwards-priming of target words by integrated perceptual representations, and perhaps also the integration (or backwardspriming) of target words into these perceptual representations in the RH. In either case, these findings provide strong support for the posited unique contribution of the RH to language comprehension involving not only the integration of perceptual information but also the attribution of meaning to this mental simulation.

Conclusion
The results of this study demonstrate the contribution of the RH to perceptual simulation intrinsic to discourse comprehension. Further, we found that the N400-effect is not necessarily produced by a negativegoing component per se. These findings advance current understanding of RH contributions to higher-level language processing in the brain and highlight the need for further investigation into the components contributing to N400-effects.

Author contributions
BG and AGM contributed to the experiment's conception, design, and programming. CAS conducted data acquisition. AGM analysed the data. All authors contributed to manuscript preparation.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.