The relationship between sign production and sign comprehension: What handedness reveals

Unlike the phonological loop in spoken language monitoring, sign language users' own production provides mostly proprioceptive feedback and only minimal visual feedback. Here we investigate whether sign production influences sign comprehension by exploiting hand dominance in a picture-sign matching task performed by left-handed signers and right-handed signers. Should all signers perform better to right-handed input, this would suggest that a frequency effect in sign perception drives comprehension. However, if signers perform better to congruent-handed input, this would implicate the production system's role in comprehension. We found evidence for both hypotheses, with variation dependent on sign type. All signers performed faster to right-handers for phonologically simple, one-handed signs. However, left-handed signers preferred congruent-handed input for phonologically complex, two-handed asymmetrical signs. These results are in line with a weak version of the motor theory of speech perception, where the motor system is only engaged when comprehending complex input.


Introduction
A growing body of evidence points to shared perceptual and motor representations in line with common coding theory (Prinz, 1984;Prinz & Hommel, 2002). Psycholinguistic theory has also begun to reject the dichotomy between perceptual and motor systems, developing more integrated theories of linguistic representations (e.g., Pickering & Garrod, 2013). Speakers make use of their comprehension system to monitor their speech production (Levelt, 1989) with the related motor muscles used during production activated during speech perception (Fadiga, Craighero, Buccino, & Rizzolatti, 2002;Watkins, Strafella, & Paus, 2003). The reverse effect, i.e., the impact of production on comprehension, is also suggested, but less understood. For example, Pickering and Garrod (2007) and Zwaan and Kaschak (2008) argue that interlocutors use covert imitation and forward modelling to make predictions, which are then used in anticipatory monitoring. The current study focuses on the impact of production systems on comprehension, making use of differences between these systems in sign languages (SLs).
While spoken languages (SpLs) are perceived via the auditory system and produced by a motor articulator (the tongue), SLs are perceived by the visual system and produced via multiple indepen-dent motor articulators (hands & face). Visual input differs from audition with the visual system providing less overlap between comprehension and production systems: visual feedback from one's own sign production is very different from visual input from another person's production, e.g., hand orientation and visual angle. Visual input during sign production is also more limited than during comprehension. Testing this, Emmorey, Bosworth, and Kraljic (2009) used a sign identification task to compare comprehension of ''other-produced" and ''self-produced" American SL signs. While signers performed well in the other-produced condition (signs showing the front of the hand in a central location, as in comprehension), they performed poorly in the self-produced condition (signs showing the back of the hand in a peripheral location, as in production) where the limited view likely impeded handshape recognition, and the peripheral location prevented detailed visual acuity. Therefore, even if signers use vision to monitor their own signing, input cannot be processed as accurately during production. The authors conclude that signers rely primarily on proprioceptive, or motor feedback, when monitoring language production.
We look at the effect of motor production systems on comprehension in British Sign Language (BSL). During production, signers employ two motor articulators: the two hands. Sign handedness refers to a signer's preferred dominant hand and signers can be either left-or right-hand dominant with no effect on meaning. Differences in handedness for SL production and comprehension sys- tems (i.e., whether two interlocutors have congruent or incongruent handedness) have the potential to inform wider integrative approaches about the relationship between motor output and linguistic perception.
Integrated models of speech perception suggest that covert production plays a role in comprehending SpLs. Skipper, van Wassenhove, Nusbaum and Small's (2007) perceptuo-motor model of speech perception posits that for SpLs, multisensory input generates the relevant motor commands in order to make motor-tosensory predictions which are then compared with the original multisensory input. In the current study we ask whether signers rely on motor feedback based on their production system during comprehension tasks. Because of handedness differences during SL production we can tease apart the relative contribution of both systems for successful comprehension. Specifically, if signers rely on motor feedback from production during comprehension, then we should see differences between right-handed signers (RHS) and left-handed signers (LHS) processing signs produced by RHS or LHS.
1.1. The phonological structure of signs and sign types SLs use both hands at once (see Vermeerbergen, Leeson, & Crasborn, 2007, for overview), and each hand can express different linguistic information concurrently when producing two-handed signs. Hand configurations in two-handed signing can be symmetrical or asymmetrical. Thus there are three basic sign types: Onehanded signs (1H), two-handed signs with symmetrical hand configurations (2HS), and two-handed signs with asymmetrical hand configurations (2HA). BSL follows universal constraints on twohanded sign forms (Battison, 1978), which prevent signs from being too complex to produce or comprehend (Sutton-Spence & Woll, 1999).
Based on the 2400 attested signs in the BSL SignBank (Fenlon et al., 2014), 45% of signs are 1H, 31% are 2HS and 24% are 2HA signs. A further estimate from the Australian Sign Language (Auslan) Corpus (Johnston & Schembri, 2006), a highly-related language to BSL, suggests that 64% of signs are 1H (Johnston, personal communication). This higher figure may be attributable to weak drop (Battison, 1974;Deuchar, 1981) where the non-dominant hand is dropped in informal conversations, as found in the Auslan Corpus, but not in formal citation forms in the BSL SignBank. The frequency of different sign types is likely driven by production constraints with frequency (from high to low; 1H > 2HS > 2HA) following a pattern of motoric economy (Lindblom, 1990). Whether or not there are differences in lexical-level processing that are driven by phonological sign type is currently unknown.
In one handshape monitoring task, Grosvald, Lachaud, and Corina (2012) found interactions between sign type and hand configuration markedness, with 1H stimuli recognised faster with marked hand configurations, but 2H stimuli (2HA & 2HS) recognised faster with unmarked hand configurations. These results suggest that perceptual complexity plays an important role in SL processing. Interestingly, there was no main effect of sign type, suggesting no differences in processing different sign types. However, the monitoring task focussing on phonology could have driven these results. The current experiment requires participants to access meaning (in a picture-sign matching task) that allows for effects of sign type on lexical-level processing to emerge. Importantly, only 1H and 2HA signs reveal signer handedness, whereas handedness is unclear for 2HS signs. Given the higher rates of right-handedness in BSL users (approximately 80% of BSL signers; Papadatou-Pastou & Sáfár, 2016), signs revealing signer handedness may be easier to comprehend when produced by RHS compared to LHS. This would be the case if signers rely only on visual comprehension systems when comprehending other signers. If, however, production systems are implicated during comprehension (e.g., with covert imitation) then there may be differences in comprehension for left and right-handers. The present study considers how the handedness of signers (both model and participant) relates to the comprehension of sign types that reveal or do not reveal signer handedness. Specifically, we investigate whether frequency of exposure to handedness, or body-type congruency based on a sign perceiver's own production system drives comprehension, and whether comprehension during a picture-sign matching task varies across sign types of varying phonological complexity.
The behaviour of LHS is crucial in determining what drives sign perception. If left-handers comprehend RHS better than LHS, it would suggest that comprehension is primarily driven by the comprehension system: during comprehension all signers are exposed most frequently to RHS. If, however, left-handers comprehend LHS better than RHS, this will provide evidence that a signer's own production system primarily drives perception. A third possibility is that both systems play a role in comprehension. This could mean left-handers are equally good at perceiving RHS and LHS; or it may result in differences across sign type, perhaps as a function of phonological complexity. Lastly, signers might show no differences in comprehending right-or left-handers, which would support signers' anecdotal claims that they do not notice differences during comprehension.

Participants
Forty-three Deaf 1 fluent BSL signers (21 female, 22 male; average age 33, range 19-59) participated in the experiment. Twenty-six were right-handed and 17 were left-handed. Eighteen participants were exposed to BSL from birth, and 21 acquired BSL non-natively (Mean age sign exposure = 8; range 3-16 years old). All participants had at least 12 years BSL exposure and rated themselves as highly fluent (<6 on a 7-point self-rating scale).

Stimuli
Experimental materials were BSL signs with corresponding black-and-white line drawings. Sign stimuli belonged to one of three categories: 1H (n = 80), 2HS (n = 80), and 2HA (n = 59, see Fig. 1). Fewer 2HA signs were included because they are the rarest sign type overall, making it a challenge to find picturable signs. 2HA were classified as signs in which the two hands differed in at least one phonological feature (handshape, movement, location). Signs where the location was only marginally different (e.g. PAR-ADE and LIFT where one hand is in front, or on top of the other) were categorised as 2HS. Because 2HS signs look almost identical when produced by RHS or LHS, they served as a control condition and we expect there to be no differences in RTs or accuracy between RHS and LHS. Sign stimuli were produced by four sign models (native BSL signers), two left-handed and two righthanded. Timing of the signs produced across the different sign models was controlled so that it did not differ significantly. Four lists were created such that each signer appears 25% of the time, with order of presentation randomised throughout.

Procedure
Participants' task was to decide as quickly and accurately as possible whether a picture followed by a BSL sign referred to the same object (answers 50% yes, 50% no) via keyboard button press with the location of the yes/no response (f/j keys) counterbalanced across participants. Each trial began with a fixation cross (400 ms), followed by the picture (1000 ms), followed by the sign video (RTs were measured from video onset; Fig. 2). Stimuli were presented using E-Prime 2.0 (Psychology Software Tools, 2012). The experiment began with 10 practice items.

Results
Four participants were excluded: one for low accuracy (<70%) and three due to data collection error. Analyses primarily tested effects of handedness (both participant and sign model hand). In all analyses we employed mixed-effects models with crossed random effects for subjects and items, fit using restricted maximum likelihood estimation, using the lme4 package (Bates & Sarkar, 2009) within R (Version 0.98.1074). We began by examining accuracy of responses, considering 'yes' and 'no' responses. Overall accuracy across all 39 participants was 96.84% (range 91-100%; see Table 1). ModelHandedness was the only significant predictor of accuracy (F(1, 0.17) = 2.39, p = 0.017) with all participants more accurate to respond to right-handed sign models.
We next turned to RT analyses considered only ''yes" answers. Two items were excluded for low accuracy (<70%) and three for unusually slow response times (50% > 2SD from participants average; 104 total items). Error trials and trials with RTs > 2.5 standard deviations from a participant's average were excluded. Average  , and Group (native/late-learners). We also included interaction terms beginning with all possible interactions. Initial models also considered handshape markedness (marked/unmarked), but no effects were found (p > 0.8) and markedness did not improve the models as a predictor of accuracy or RTs. Further, lexical frequency and age of acquisition (based on norms) did not differ among the sign types and did not improve the models. For the sake of brevity, we present only the F tests from the LMER results here (type III Wald F-tests with Satterthwaite approximation for degrees of freedom; See Table 2).

Discussion
Overall, both right-and left-handed native and non-native signers responded faster when they saw a RHS, regardless of sign type. As mentioned in the introduction, SL input most often reveals right-handedness, due to the high prevalence of RHS coupled with the high frequency of signs that reveal handedness (1H & 2HA signs). Comprehension speed in our experiment was influenced by this frequency. Specifically, how often sign perceivers compre-hend a RHS producing a sign showing a signer's handedness affects processing speed regardless of whether the sign perceiver is right or left-handed. Thus, overall faster RTs in identifying signs produced by a RHS points to the dominant role of the comprehension system in sign comprehension.
However, the comprehension system alone cannot explain the results. Left-handed participants were faster to respond to 2HA signs produced by LHS compared to RHS, while this was not the case for right-handed sign participants who were faster for RHS. This body-type congruency effect suggests a role of the production system in sign comprehension in line with a perceptuo-motor model of speech perception: left-handers respond faster to LHS who share the same motor system during production. The question then is: Why is the production system implicated when processing one type of sign (2HA) but not another type (1H)?
Based on an overview of the literature, Hickok, Houde, and Rong (2011) argue that motor production systems may in fact be engaged only part of the time during comprehension. They argue against a strong version of the motor theory of speech perception, instead concluding that the motor system 'modulates' the perceptual system to some extent. Further, based on the studies they review, this modulation may be activated in the face of more difficult processing input such as noisy speech. Our findings support the implication of production systems under difficult processing conditions by showing a lack of motor system engagement for simpler 1H signs, and evidence for motor activation when processing more phonologically complex input from 2HA signs. Put simply, the more 'easy' and frequent 1H signs do not appear to engage the motor system, whereas the 'complex' and less frequent 2HA signs do.
For SpLs, there are no direct parallels for RHS and LHS, or phonological differences based on the presence or absence of multiple articulators (i.e., 1H, 2HS and 2HA signs). SpLs only make use of one articulator, and consequently words cannot be articulated 'symmetrically' or 'asymmetrically'. Nonetheless, the high frequency of input from signers with a certain physical characteristic such as left-handedness might be compared to comprehending speech with an unfamiliar accent. Research from regional accents in French show evidence of temporary processing costs claimed to be due to an adjustment mechanism that normalises accented speech (Floccia, Goslin, Girard, & Konopczynski, 2006). It is possible that signers may also unconsciously adjust to left-handed input via a similar mechanism. Interestingly, neuroimaging studies of face-to-face hand movement imitation (e.g., Koski, Iacoboni, Dubeau, Woods, & Mazziotta, 2003) suggest mirror imitation with the ipsilateral hand is preferred, implying activation in the contralateral cortex. This contrasts with the faster RTs seen by lefthanded signers to congruent-handed 2HA signs. Signers may therefore behave differently, given handedness preference and/or mental rotation in sign languages.
Finally, lexical frequency and hand configuration (marked or unmarked) did not affect processing in the current experiment, likely because they were relatively balanced among the items. However, there was a consistent effect of sign type such that participants responded fastest to 2HS signs (where 1H input is essen- tially duplicated), followed by 1H signs, and slowest for 2HA signs. Interestingly, this pattern of results is not the same as has been claimed for sign production with 1H signs being the easiest to produce motorically (but see Cheek, Cormier, Repp, and Meier (2001) for errors in acquisition). While it may be easier to produce a 1H sign, comprehending 2HS signs involves perception of the same phonological input doubled, which likely leads to greater ease during comprehension. Thus the results provide evidence for differences in processing costs between comprehension and production.

Conclusions
This research provides evidence for sign comprehension being driven both by the comprehension system and by the production system. For asymmetrical sign types, where handedness is apparent, LHS perform in different ways depending on whether signs are 1H or 2H. Differences in phonological complexity of these sign types may influence whether production or comprehension drives sign comprehension, and hence whether body-type congruency effects or frequency effects are seen. These results are in line with a weak version of the motor theory of speech perception whereby the motor system modulates sign comprehension when processing complex or difficult input. However, facilitation effects of covert sign production otherwise give way to frequency effects of input showing a signer's right-handedness.