External location of touch is constructed post-hoc based on limb choice

When humans indicate on which hand a tactile stimulus occurred, they often err when their hands are crossed. This finding seemingly supports the view that the automatically determined touch location in external space affects limb assignment: the crossed right hand is localized in left space, and this conflict presumably provokes hand assignment errors. Here, participants judged on which hand the first of two stimuli, presented during a bimanual movement, had occurred, and then indicated its external location by a reach-to-point movement. When participants incorrectly chose the hand stimulated second, they pointed to where that hand had been at the correct, first time point, though no stimulus had occurred at that location. This behavior suggests that stimulus localization depended on hand assignment, not vice versa. It is, thus, incompatible with the notion of automatic computation of external stimulus location upon occurrence. Instead, humans construct external touch location post-hoc and on demand.


17
Spatial perception and actions rely on multiple spatial codes, often 18 associated with different reference frames. For instance, the accuracy of 19 pointing or reaching with an arm or finger to a visual target depends not only on 20 the position of target relative to gaze (Fiehler et al., 2011;Thompson et al., 21 2014), but also on salient world-centered landmarks (Schütz et al., 2013). 22 Similarly, judgment of visual location during whole-body movement is 23 influenced by a target's position relative to gaze, as well as by the location of 24 the target relative to the body (Tramper & Medendorp, 2015). 25 In touch, too, space is coded in several reference frames. Touch 26 activates specialized sensory receptors embedded in the skin, and the 27 arrangement of the peripheral sensors is reflected in the homuncular 28 organization of primary somatosensory cortex (Penfield & Boldrey, 1937;Roux 29 et al., 2018), referred to as a skin-based or somatotopic layout. However, 30 because our body can take various postures, the stimulus location in space -31 often referred to as its external location -must be derived by combining skin 32 location and body posture, a process termed tactile remapping (Heed, Buchholz, 33 et al., 2015). Indeed, there is evidence that external tactile locations can be 34 coded in a gaze-centered reference frame (Harrar & Harris, 2010;Mueller & 35 Fiehler, 2014a, 2014b), but also relative to anchors such as the head, torso, 36 and hand (Alsmith et al., 2017;Heed et al., 2016). 37 It is less clear, however, according to which principles these different 38 spatial codes are employed. Both bottom-up features such as the availability of 39 sensory information (Bernier & Grafton, 2010)  consensus that each spatial code can have more or less influence depending 48 on the specific situation, it is currently not known whether all putative codes are 49 always constructed, or whether they are only computed based on demand. 50 For touch, it has been suggested that the construction of spatial location 51 is an automatic process, implying that any tactile input is remapped into an 52 external code, irrespective of its relevance (Heed & Azañón, 2014;Röder et al., 53 2004). The most common experimental manipulation underlying this claim is 54 limb crossing. Crossing, say, a right arm over to the left side of space leads to 55 different skin-based (here: right body side) and external (here: left side of 56 space) spatial codes of a tactile stimulus delivered to the right hand. A task-57 irrelevant tactile stimulus delivered to a crossed right hand accelerates visual 58 discrimination in the right visual field if it precedes the visual target stimulus by 59 60 ms, but on the left side if it leads by 180 ms or more 60 2008). Thus, responses to visual targets were faster after anatomically 61 congruent tactile cues (e.g., tactile stimulus on crossed right hand, visual target 62 in right hemifield) at short cue-stimulus intervals, but after externally congruent 63 tactile cues (e.g., tactile stimulus on the left hand crossed over to the right side, 64 visual target in right field) at long cue-stimulus intervals. Such effects are 65 usually interpreted as evidence that tactile remapping -the precise 66 computation of the external tactile stimulus location -is automatic and forms 67 the basis for the performance enhancement at this external location. 68 The same conclusion has also been drawn from results obtained with 69 the tactile temporal order judgment (TOJ) task; in this task, participants report 70 which of two successive tactile stimuli, each presented to a different body part 71 -typically the two hands -occurred first (Heed & Azañón, 2014;Shore et al., 72 2002; Yamamoto & Kitazawa, 2001). When the time interval between the two 73 stimuli is short, participants sometimes choose the wrong stimulus. Notably, 74 stimulus confusion is much more prominent when the arms are in a crossed 75 than uncrossed posture. This is surprising because the TOJ task asks about 76 the identity of the touched limb, and, in theory, it would be irrelevant to this 77 question where the hand was in space. That limb crossing, nevertheless, affects TOJ implies that posture cannot be strategically ignored, but is 79 automatically incorporated into the hand assignment. 80 Several explanations have been put forward to account for crossing 81 effects in tactile localization. First, it has been suggested that touch location, 82 once it is remapped, is retained only in an external spatial code, and the original 83 skin location is discarded in the process. To report which body part has been 84 touched, the brain must then reversely determine which limb was located at the 85 computed external location at the time of the touch (Kitazawa, 2002;Kitazawa 86 et al., 2008;Yamamoto & Kitazawa, 2001). We refer to this suggestion as the 87 space-to-limb reconstruction hypothesis. When applied to errors in the TOJ task, 88 this hypothesis implies that participants correctly remap the two tactile stimuli 89 into external space, but then reconstruct erroneously which hand was at the 90 first spatial location. 91 A second explanation assumes that TOJ errors reflect the conflict 92 between different codes used for stimulus location. When the limbs are 93 crossed, skin-based and external spatial codes point to different sides of space, 94 and this conflict must be resolved, a process that takes time and is error-prone 95 (   The response, thus, contained two components: the hand to which the first 195 stimulus was assigned, and explicit spatial localization of this stimulus. 196

Hand assignment 197
In a first step, we verified that hand assignment in the TOJ task was modulated 198 by hand crossing and timing of stimuli relative to the movement (Heed,Möller,199 Fig. 2, green vs. 236 magenta colors) increased. For instance, for movements from an uncrossed to 237 a crossed posture, TOJ performance was better during the first two movement 238 phases, that is, when the hands were still uncrossed, than during the last two 239 movement phases, that is, when the hands were crossed.

Explicit tactile localization is directed towards the assigned hand 292
We have so far assessed performance in trials in which participants had 293 made a correct TOJ hand assignment (referred to as correct TOJ trials from 294 hereon). We now turn to localization errors in incorrect TOJ trials. These errors 295 allow differentiating between the three hypotheses about how participants 296 determine stimulus localization in tactile decision paradigms (see Fig. 1D-F). 297 We first turn to the space-to limb reconstruction hypothesis. It posits that 298 tactile perception takes place in space rather than on the body; thus, a limb 299 assignment entails computing which limb was at the first spatial location. Thus, 300 in our task, responses with the incorrect hand would result from assigning the 301 incorrect hand to the correct spatial location of the first tactile stimulus (see Fig.  302 1D). Accordingly, the assigned, incorrect hand should be directed to the 303 location at which the stimulus of the other, correct hand had occurred, and the 304 reported stimulus location in incorrect TOJ trials should scatter around the 305 movement trajectory of the correct hand. Contrary to this prediction, participants 306 consistently pointed to locations scattered around the movement trajectory of 307 the assigned, incorrect hand, indicating that the chosen stimulus had been 308 perceived on the incorrect hand (see Fig. 4 for the localization responses of the 309 participant with the largest variability in localization errors). Thus, localization 310 behavior did not support the implication of the space-to-limb reconstruction 311 hypothesis that the correct external spatial location is simply assigned to a 312 wrong limb.   Table 4). 494

Explicit stimulus localization in space 495
Complementing the findings from Experiment 1 and further corroborating the 496 time reconstruction hypothesis, localization errors of the incorrect TOJ trials 497 largely overlapped with the localization errors of correct TOJ trials at time 1 for each of the four SOAs (see Fig. 7) and for each subject and posture condition 499 (see Supplementary Fig. 2-4). as for all tested SOAs between the two tactile stimuli. TOJ errors, thus, did not simply reflect temporal confusion of two stimuli; instead, localization in TOJ 602 error trials marks the computation of tactile stimulus location based on correct 603 stimulus timing and movement information of a (correctly or incorrectly) implied 604 body part. Accordingly, limb crossing affected hand assignment, but not 605 stimulus localization. 606 The pattern of hand assignment errors was in line with previous studies: 607 Participants made more TOJ hand assignment errors in conditions that involved 608 hand crossing than in conditions in which the hands were uncrossed (see suggesting that arm posture during stimulation did not affect localization 617 responses (see Fig. 4, 5). In particular, localization errors exhibited comparable 618 spatial biases over time in uncrossed and crossed conditions. Furthermore, 619 localization error scattered around the chosen hand was not biased towards the 620 other hand (see Fig. 4), an effect one might have expected if, like hand 621 assignment, spatial localization was subject to weighted influence of the tactile 622 stimulus's anatomical origin as coded by a body-based reference frame. 623 The dissociation between TOJ hand assignment and localization 624 responses indicates that the two phenomena do not reflect the same process. blocks of 50 trials in pseudo-randomized order (see Fig. 1A). In each trial, a 800 tone instructed the movement start. At a random time (presented between 50-801 800 ms after the tone, drawn from a square distribution) before, during, or after 802 the movement, two tactile stimuli were applied, one to each hand, at an SOA of 803 110 ms; the left-right order of stimuli was pseudo-random. Upon movement 804 completion, participants moved the index finger that they had perceived to have 805 been stimulated first to the location of the first stimulus on the table. The hand 806 remained in this location until a tone, presented 2.5 s after the initial movement 807 cue instructed them to lift the index finger; this finger lift was used to identify the 808 response hand during trajectory analysis. Subsequently, participants 809 repositioned the hands to their start locations. We acquired 300 trials of each 810 posture combination. To compensate for obstruction of motion tracking 811 markers, we acquired more trials for 2 participants in the uncrossed to crossed 812 posture and vice versa movement condition. The experiment took 813 approximately 4 hours, split in two-hour sessions held on different days. 814 Practice trials were included on each day before the experiment started until the participant had understood, and felt confident with, the task. In total we 816 acquired 14.776 trials. 817 1 Stimulus Task: The procedure was identical to the 2 Stimulus task except that 818 participants only received one stimulus at either hand and than indicated the 819 perceived location with the respective index finger. Participants performed 300 820 trials in each posture combination split in blocks of 50 trials. In 99,5% of the 821 trials participants were using the correct arm when localizing the stimulus. 822 Analysis 823 Data preprocessing. Start and end of the movement were determined based on 824 a velocity threshold of 5 cm/s. We interpolated missing motion tracking data, 825 for instance due to obstruction when the hands passed each other or due to 826 rotation of the hands, using splines, with the restriction that movement onset 827 and offset could be determined. Trials were discarded when (1)  Localization error. We calculated the localization error, that is, the difference 839 between the true location of the index finger at the time of stimulation and the 840 reported location, that is, index finger pointing location just before finger lifting. incorrect responses in our TOJ task (Jaeger, 2008). Furthermore, (G)LMM and 884 are robust against missing data and account for differences in trial numbers 885 across conditions, as present in our data. All reported statistics were computed 886 using type 3 sums of squares, as implemented in afex. For the random structure 887 of LMM and GLMM for TOJ analysis, we included only random intercepts, 888 because models did not reliably converge when random slopes were included. The brms R package uses STAN as backend. We ran LMM to estimate 897 the 95% interval of the intercepts in the different time shift models. We 898 compared Bayesian models using the loo_compare() function of the loo R 899 package. This function uses leave-one-out cross-validation to compare models 900 by assessing the models' predictive density when each data point is omitted 901 from fitting . 902

Experiment 2 903
Apparatus, task and procedure 904 Kinematic data of the fingers was recorded using an optical motion capture 905 system (Visualeyez II VZ4000v, Phoenix Technologies Inc, Vancouver, BC, continuous, and synchronous movements (6.8%). In total we removed 18.2% 937 of the trials. 938