Contingent sounds change the mental representation of one’s finger length

Mental body-representations are highly plastic and can be modified after brief exposure to unexpected sensory feedback. While the role of vision, touch and proprioception in shaping body-representations has been highlighted by many studies, the auditory influences on mental body-representations remain poorly understood. Changes in body-representations by the manipulation of natural sounds produced when one’s body impacts on surfaces have recently been evidenced. But will these changes also occur with non-naturalistic sounds, which provide no information about the impact produced by or on the body? Drawing on the well-documented capacity of dynamic changes in pitch to elicit impressions of motion along the vertical plane and of changes in object size, we asked participants to pull on their right index fingertip with their left hand while they were presented with brief sounds of rising, falling or constant pitches, and in the absence of visual information of their hands. Results show an “auditory Pinocchio” effect, with participants feeling and estimating their finger to be longer after the rising pitch condition. These results provide the first evidence that sounds that are not indicative of veridical movement, such as non-naturalistic sounds, can induce a Pinocchio-like change in body-representation when arbitrarily paired with a bodily action.

(1) the represented finger would become longer in the rising pitch condition, while a shortening of the finger would not necessarily occur in the descending pitch condition given that illusory bodily reductions are harder to induce than bodily extensions 36 ; (2) the change in the represented finger length would come from the illusory movement of the pulled fingertip raising (or descending) in space induced by the sound; (3) the change in the represented finger length would come from the rising (or descending) sound being interpreted as an overall change in size of the pulled finger.
The three predictions are compatible, though each could hold independently of the other two.

Experiment 1.
Estimates of fingertip position, knuckle position and finger length. For each trial, the estimated knuckle position was subtracted from the estimated fingertip position in order to calculate the estimated finger length. For each estimate type (estimated finger length, fingertip position and knuckle position), data from trials exceeding three standard deviations from the mean group value were excluded. For those participants with only one trial per condition excluded (two participants), data for that trial was replaced by the mean value of the other nine trials for that condition. Those other (two) participants for whom more than one trial per condition had been eliminated were excluded from all analyses. Data analyses were performed then on the remaining data of 22 female participants (mean age = 20.05, range = 18-23). The mean estimates are displayed in Fig. 2. Individual z-scores of these behavioural data were calculated in order to achieve normality (normality checked was performed with Shapiro-Wilk tests) and data from the 10 repetitions for each condition were averaged. These data on estimates of fingertip position, knuckle position and finger length were then submitted to three separate within-subjects analyses of variance (ANOVA) with 'sound' ('ascending' , 'descending' or 'constant') as factor. These analyses showed a significant main effect of sound for finger length (F(2,42) = 4.80, p = 0.013, η 2 p = 0.19); the effect size of the mean difference between conditions is large according to Cohen's rule 37 . No effects of sound emerged for knuckle or fingertip position (see full summary of F-values and p-values in Table 1). Significant effects were further investigated with independent-samples t-tests, with the significance alpha level adjusted for multiple comparisons using Bonferroni correction. These t-tests revealed larger finger length estimates for the 'ascending' sound, compared with the 'descending' sound, t(21) = 2.77, p = 0.011, Cohen's d = 1.07, and with the 'constant' sound, t(21) = 2.61, p = 0.016, Cohen's d = 0.83. For both comparisons, the effect size of the difference between the two means is large according to Cohen's rule 37 . No difference on estimated finger length was found between 'descending' and 'constant' sounds.
Visual templates of index finger. An ANOVA on aligned rank transformed data, followed by Wilcoxon signed ranked tests, was used to explore the effect of 'sound' . When asked to select a visual drawing to describe the subjective feeling of their finger when listening to the sound (see Fig. 2 -bottom panel), participants selected significantly larger visual matches to their index finger after the 'ascending' sound (median ratio was 8:7, range was 6:7 to 10:7) as compared to the 'constant' (Z = 3.62, p < 0.001; median ratio was 7:7, range was 4:7 to 8:7) and 'descending' (Z = 2.93, p = 0.003) sounds (F(2,63) = 9.11, p < 0.001; median ratio was 7:7, range was 3:7 to 10:7). No difference was found between 'descending' and 'constant' sounds.
Other subjective results. Questionnaire data were analysed with ANOVAs on aligned rank transformed data comparing the three sound conditions. Significant effects were further investigated with Wilcoxon tests, with the significance alpha level adjusted for multiple comparisons. For all statistical tests alpha level was set at 0.05, 2-tailed. Subjective reports (see full questionnaire data and analyses in Table 2) revealed that participants felt that their finger was longer to a larger extent in the 'ascending' sound conditions than in the 'descending' sound conditions (Z = 2.67, p = 0.008); they also felt that their finger was descending to a lesser extent (Z = 3.03, p = 0.002). Furthermore, in the 'descending' sound conditions participants felt that their finger was shorter to a larger extent Participants were asked to hold their index finger still and straight. On each trial, the pulling of the finger was detected by a pressure sensor and simultaneously triggered a 2000-ms pure tone ('ascending' , 'descending' or 'constant' in Experiment 1, 'ascending' or 'descending' in Experiment 2) that participants listened to through headphones. After the sound ended, the task was to estimate the position of the fingertip and knuckle using the apparatus shown in the right circle. In Experiment 1 participants kept their index fingertip pointing upwards during all the experiment, which included 10 repetitions of each sound type. In Experiment 2 the position of the hand was varied between two experimental blocks (each block with 10 repetitions of each sound type), so that participants either kept the fingertip pointing upwards or downwards, as shown in the figure (note that the ruler is displayed for illustration purposes, but that it was blacked out on the side facing the participant). than in the 'constant' sound conditions (Z = 2.63, p = 0.008). Note that results of correlational analyses between estimates and subjective reports are provided as supplementary material.

Experiment 2.
Estimates of fingertip position, knuckle position and finger length. Similar data analyses to the ones in Experiment 1 were performed. For those participants with only one trial per condition excluded (six participants), data for that trial was replaced by the mean value of the other nine trials for that condition. Those other participants (two participants) with data missing from more than one trial per condition were excluded from all analyses. Data analyses were performed then on the data of 22 female participants (mean age = 21.7, range = 19-26). 10 had the 'upwards' position first, and 12 hand the 'downwards' position first. The mean estimates are displayed in Fig. 3.
Individual z-scores of these behavioural data were calculated in order to achieve normality (normality checked was performed with Shapiro-Wilk tests) and data from the 10 repetitions for each condition were averaged. These data on estimates of fingertip position, knuckle position and finger length were then submitted to three separate 2 × 2 ANOVAs with 'finger position' ('upwards' or 'downwards') and 'sound' ('ascending' or 'descending') as within-subjects factors and block order ('upwards first' , 'downwards first') as a between-subjects factor. No effects of block order emerged. The analyses on the finger length data showed a significant main effect of sound (F(1,20) = 7.48, p = 0.013, η 2 p = 0.272), revealing larger finger length estimates for the 'ascending' sound, compared with the 'descending' sound; the effect size of the mean difference between conditions is large according to Cohen's rule 37 . Neither the effect of finger position or its interaction with sound reached significance (see full  Table 1. Results from statistical tests on estimates of fingertip position, knuckle position and finger length in Experiment 1. Results from ANOVAs comparing the effects of sound in the three conditions are in the second column. Significant effects were further investigated with independent-samples t-tests (with correction for multiple comparisons α = 0.017) and are displayed in columns three to five. Significant differences are marked in bold font.
summary of F-values and p-values in Table 3). The analyses on fingertip position showed a significant main effect of finger position (F(1,20) = 506015.52, p < 0.001, η 2 p = 1.00), revealing that fingertip position was estimated to be at a higher position in the 'upwards' than in the 'downwards' position; this effect was expected because the actual fingertip position indeed changed across 'upwards' and 'downwards' blocks while the actual knuckle position was kept fixed. No effects emerged for knuckle position and no significant two-way interactions between sound and finger position emerged for any of the variables.
Other subjective results. Questionnaire data were analysed with ANOVAs on aligned rank transformed data. These subjective reports (see full questionnaire data and analyses in Table 4) confirmed again that, in the 'ascending' conditions participants felt that their finger was longer (F(1,84) = 11.44, p = 0.001), that it was rising (F(1,84) = 11.33, p = 0.001) and that it stretched (F(1,84) = 7.73, p = 0.007) to a larger extent than in the 'descending' conditions, and they felt that their finger was shorter (F(1,84) = 5.08, p = 0.027) and that it was descending to a lesser extent (F(1,84) = 11.60, p = 0.001). Further, in the 'upwards' conditions participants felt, to a larger extent than in the 'downwards' conditions, that their finger was rising (F(1,84) = 7.69, p = 0.007) and they felt to a lesser extent that their finger was descending (F(1,84) = 23.50, p < 0.001). Results of correlational analyses between measures are provided as supplementary material.

Discussion
Results demonstrate an 'auditory Pinocchio' effect, with participants estimating the length of their finger to be longer after a two-second rising pitch sound, accompanied by pulling of their fingertip, as contrasted with either a descending pitch or with a constant tone. Crucially, though there were changes in the absolute estimated location of the fingertip, these do not explain the whole effect of sound, indicating an illusory finger extension independent of the illusory movement. Experiment 2 shows that the effect holds across hand positions, and upward or downward direction of finger pulling. This confirms that the contrast between the rising and descending sounds, and not the direction of the pulling movement exerted by the other hand, drives the illusion. Changes in body-representation were also evidenced in subjective reports. Participants selected drawings of a significantly longer finger as a match for their own after listening to the 'ascending' sound, as compared to the other sounds. The questionnaire corroborated these results.    (1 to 7). Results from ANOVA on aligned rank transformed data comparing three conditions are in the fifth column. Significant differences are marked in bold font. Significant tests were followed by Wilcoxon signed ranked tests that are presented in columns six-eight (with correction for multiple comparisons α = 0.017).
Besides proposing an embodied analogue to the film adaptations of Pinocchio, where rising pitch sounds are used to accompany the extension of the character's nose, the study also contrasts with the dominant reliance on vision in studying the plasticity of body-representations. Audition is a newcomer in this field, with recent studies documenting its role in the shaping and updating of one's body-representations. While existing studies all draw on naturalistic sounds [8][9][10][11][12][13] , similar to those produced when one moves or interacts with physical objects, our results demonstrate for the first time that the effects of sounds on represented body size are not limited to such cases. Moreover, we demonstrate that new audio-tactile contingencies can affect represented body size in a relatively short amount of time. Other illusions introducing visuo-tactile conflict with the human hand require longer exposure to stimuli to achieve significant effects, such as in the case of the rubber-hand illusion, which arises approximately after 11 seconds of stimulation 38 (note that most studies on the rubber-hand illusion employ periods of stimulation of at least 60 seconds, e.g. refs 6,7 ). These visuo-tactile illusions are also partly constrained by the congruence between the visual stimuli and one's previous experience of one's own body 39 . By contrast the change in pitch used here is not formerly associated to one's body or indicative of a veridical displacement of one's body in space. The explanation of the effect probably needs to appeal to the robust mappings that exist for dynamic pitch changes. As mentioned earlier, two correspondences, between dynamic pitch changes and vertical motion, and between change in pitch and change in size, affect behaviour automatically and universally 15,17,19,27,40 . The former mapping has been mostly demonstrated across audition and vision, but it also applies across audition and touch 28 . Recent findings suggest that the ear-filtering properties and sound localization processes are fine-tuned to mirror a frequency-elevation mapping found in natural auditory scenes and to change with proprioceptive and vestibular cues as head position changes 19 . When it comes to perceiving motion in the vertical plane, spectro-temporal cues play a bigger role than binaural cues, and the filtering properties of the pinna become relevant to detect differences in frequency 41 . Neuroimaging studies have revealed that rising/falling pitch operate at an intermediate level of the cortical hierarchy with respect to spatial words (such as 'LEFT' or 'UP') and to sounds actually moving: while moving sounds activate audio-visual motion areas (hMT+/V5+) and spatial words activate the right intraparietal sulcus (a higher-level convergence region) the effects of rising and descending pitch could be seen in both areas 42 . The activated superior parietal regions that are implicated in spatial processing 42 overlap with multisensory parietal areas integrating somatosensory, visual and auditory signals to form body-representations 43 , which suggests an interaction between sound localization driven by changes in pitch and internal models of body-representation.
We show that a changing pitch led to a change in estimated finger length rather than a change in estimated vertical position, showing that body-representation of size is updated to conform to the proprioceptive information provided by the immobile forearm. These findings are distinct but compatible with the second mentioned correspondence between rising pitch and an overall increase in visual size 34 . Whether the present "auditory Pinocchio" effect is driven by touch and/or proprioception is open to further testing. In our experiments, we asked participants to pull their right index finger, rather than merely apply pressure to it with the left hand, to minimise any multisensory conflict that might cancel out the Pinocchio effect. In the literature, dynamic sound stimuli (rising/descending pitch) have been shown to be effective at inducing crossmodal effects when paired with dynamic visual stimuli 27,34,44,45 . A dynamic pulling action was therefore chosen here as the closest analogue. A further open question, therefore, is whether the pulling action is necessary to induce the effect, or whether mere pressure on or contact with the fingertip paired with a synchronous increasing pitch would be sufficient. A question also opened for further testing is the stronger effects for the rising than for the falling pitch that our study revealed, and which other studies on spatial crossmodal cueing of pitch on visual targets have similarly found 46,47 . Apart from this reported asymmetry on the crossmodal cueing of pitch, in the literature on sensory-driven changes in represented body size there are many fewer reports of illusory body shrinkage than of body expansion 36 (although see refs 48,49 ). Some authors have suggested that this asymmetry may reflect the fact that people are more used to experience the enlargement, as opposed to the shrinkage, of their body parts, for instance, during normal growth 36 . One further open question is whether the reported effects would be stronger if tactile and auditory stimuli would be spatially coincident, which could be achieved by using spatialization techniques over headphones or by using a loudspeaker next to the finger. We expect that the effects we found would hold when using loudspeakers as previous studies looking at the mapping between sound pitch and spatial elevation have used either headphones 25,42 or loudspeakers 22 . The present method produced sizable effects 37 Table 4. Median (Range) for questionnaire data in Experiment 2. Results from ANOVA on aligned rank transformed data looking at the effects of finger position, sound condition and their interaction are in columns sixth to eight. Significant effects are marked in bold font.
the effect held across two experiments, with two different experimenters (who were blind to the experimental conditions) and two different sets of participants, providing support to the robustness of the effect. However, future studies may consider using some sort of digital marker or using video coding to avoid the experimenter lag and increase sensitivity.
All these effects could be tested with various populations on other body parts, where sound might generate changes in length, width or both. The "auditory Pinocchio" effect reveals the strength of the auditory modulation of body-representation, and opens new possibilities to use arbitrary sounds in therapeutic or virtual settings. Sound feedback is increasingly seen as promising in such settings, especially when an individual is on the move or when looking at body parts may be detrimental to the rehabilitation process. This is the case, for instance, of rehabilitation of poor proprioception during gait or other movements in the elderly, in people with Parkinson's Disease, in people who suffered a stroke or in people with chronic pain, who sometimes are reluctant to look at their affected body part [50][51][52][53][54][55] . Our study here adds to this by suggesting that auditory-driven strategies do not need to rely on visual or muscle stimulation, nor be constrained to actions that naturally produce sound (e.g., footsteps). Apparatus and stimuli. Three experimental auditory stimuli were created on Audacity software and consisted of pure tones (2000-ms duration and 44.1-kHz sample rate) of increasing ('ascending' tone: 700 to 1200 Hz), decreasing ('descending' tone: 700 to 200 Hz) or constant ('constant' tone: 700 Hz) frequency. We adopted the frequency ranges of the stimuli used by Deroy and colleagues 28 to study audio-tactile correspondences between auditory changes in pitch and tactile direction of movement. It should be noted that while the chosen frequency ranges differ in their starting/ending frequencies and their average frequency, these parameters are less significant than the direction of the frequency change in terms of mapping with changes in spatial elevation, as shown by Mossbridge and colleagues in a previous spatial cueing study 56 . Mossbridge and colleagues 56 tested various frequency ranges differing in the initial, average and final frequency and found that these parameters had no effect on the robust cross-modal effects of ascending and descending frequency changes on guiding spatial attention. An additional auditory stimulus consisting of a pure tone ('anchoring tone': 250-ms duration, 700 Hz) was used in the "anchor" session that participants performed prior to the experimental session, as explained below. A 10-ms onset/offset ramp was applied to the auditory stimuli to prevent clipping. For all auditory stimuli sound level was set at 60 dBA.

Methods
A force-sensitive resistor (FSR; 4 mm-diameter active sensing area) attached to the participant's right index fingertip was used to detect the participant's finger pulling action and trigger the auditory stimulation. The FSR was connected to an Arduino Uno microcontroller linked to a computer. Presentation ® software was used to control the stimulus delivery and to record the participant's responses.
The experimental setup is illustrated in Fig. 1. This setup consisted of a transparent plastic panel (30 × 42 cm) attached on the bottom and right side to a metal stand that has a 2 × 2 × 50 cm vertical metal bar. Participants were seated in a chair and wore a pair of closed headphones with high passive ambient noise attenuation (Sennheiser HDA 300). They were asked to hold their index finger still and straight, fingertip pointing upwards, with the dorsum of the finger pressed against the transparent panel and the right side of the finger pressed against the metal bar. Their right arm rested in a cardboard ramp placed under the hand. Their left elbow rested on foam cushions to avoid fatigue during the experimental session in which they used the left hand to pull their right index finger. A black cloak was used to hide the hand from the participant's view. This cloak was attached to the participant's neck and covered the plastic frame, ramp and hand. A coloured clip was fixed on the top centre of the plastic panel (over the cloak) and served as the centre fixation point for participants.
A specially designed apparatus was used to collect participants' estimates of their fingertip and knuckle position. This apparatus consisted of a 50-cm ruler fixed on the right side of the metal bar, parallel to it and 10 cm away. The ruler was blacked out on the side facing the participant. Two horizontal clips were mounted on the ruler, each of them with a red dot which served as visual points that the participant used to mark their felt fingertip and knuckle positions, as explained below. The two horizontal clips on the ruler were initially positioned at the heights of 10 cm and 50 cm. The height of participants' index finger knuckle with respect to this ruler was approximately 25 cm, given that the cardboard ramp kept the hand elevated.
Experimental procedure. Verbal and written instructions about the tasks were given to participants at the beginning of the session. First, participants were asked to complete the "anchor" task, which consisted of pressing their right index fingertip with their left hand twenty times, an action that triggered the 'anchoring' tone on each occasion. Note that during this task, participants were not exposed to the 'ascending' or 'descending' tones. Having completed the "anchor" task, participants were asked to complete the experimental block.
In the experimental block, participants were required to look straight at the fixation point and to perform the simple action of pulling their right index fingertip using their left hand, while keeping the right index finger in a fixed position (straight and pressed against the plastic panel and metal bar). The pulling action triggered one of the three experimental tones ('ascending' , 'descending' or 'constant'). Participants were asked to keep pulling their finger until the sound they heard was over. They could then relax their left hand, but were asked to keep holding their right finger with the left hand. They were then asked to estimate the position they felt their right fingertip and knuckle to be, by having the experimenter adjust the two visual points placed on the ruler clips. This was done by the experimenter moving the top clip downwards, at a constant speed, until the participants indicated with a "stop" signal that the upper visual point had reached the fingertip position. The experimenter then moved the bottom clip upwards, at a constant speed, until the participants indicated with a "stop" signal that the lower visual point had reached the knuckle position 57,58 for similar procedures. The experimenter recorded the fingertip and knuckle positions using the measurements on the back of the ruler, rounding to the nearest 0.5 cm (e.g. ref. 59 ). The clips were repositioned after each trial. After one practice trial (with 'constant' tone), the participant was asked to repeat the task for thirty subsequent trials. Each experimental tone ('ascending' , 'descending' , 'constant') was presented ten times. The order of trials was randomized across participants. After fifteen trials, participants were given the option to have a short break before continuing with the task.
After the experimental block was completed, participants were asked to repeat the task of pulling their finger while listening to a tone for three more trials, one trial for each sound condition, with the presentation order randomized across participants. Participants completed a questionnaire after each trial. This questionnaire assessed their subjective experience during the finger-pulling task. The questionnaire contained thirteen statements, adopted from our previous studies 10,13 . The list of statements is presented in Table 2. Participants rated their level of agreement with the statements using a 7-item Likert scale, ranging from 1 (strongly disagree) to 7 (strongly agree), with 4 referring to "neither agree, nor disagree". In addition to these statements, participants were presented with a range of 9 figures representing finger length adapted from ref. 4 . Each figure showed a prototypical whole hand with the index finger selectively shrunk or elongated. Participants were asked to choose one of the figures to describe the subjective feeling of their finger when listening to the sound. The ratio of finger length was varied from 3:7 to 11:7 of the width of the pictured hand.
Apparatus, stimuli and experimental procedure. The experimental auditory stimuli, apparatus and experimental procedure were identical to those in Experiment 1, with the following exceptions. First, in this case only the 'ascending' and 'descending' tones were used. Second, in this case the ruler was placed on the left instead of the right side of the plastic panel. Third, a 1 cm-thick metal bar was positioned 10 cm to the right of the ruler and attached to the surface of the plastic panel on the side facing the participant. Finally, participants did not rest their right arm in a cardboard ramp placed under their hand, but instead kept the hand elevated during the full experimental block.
Depending on the experimental block, participants either kept the fingertip pointing upwards or downwards (see Fig. 1). In the 'upwards' block, participants were asked to place their hand on the left side of the metal bar and to keep their index finger held straight, fingertip pointing upwards, with the dorsum of the finger pressed against the transparent panel and the right side of the finger pressed against the metal bar. In the 'downwards' block, participants were asked to place their hand on the right side of the metal bar and to keep their index finger held straight, fingertip pointing downwards, with the dorsum of the finger pressed against the transparent panel and the left side of the finger pressed against the metal bar. The height of participants' index finger knuckle was fixed to be the same in the upwards and downwards blocks (height with respect to the ruler was approximately 25 cm).
As in Experiment 1, participants were first asked to complete the "anchor" task, in which they pressed their right index with the fingers of their left hand twenty times, an action that triggered the 'anchoring' tone. Next, participants were asked to complete two experimental blocks, one with the 'upwards' finger position and the other with the 'downwards' finger position. In each block each experimental tone ('ascending' or 'descending') was presented ten times, resulting in twenty trials per block. The order of trials was randomized across participants. Whether the 'upwards' block or the 'downwards' block was presented first was counterbalanced between participants. Participants were given the option to have a short break between blocks.
After both experimental blocks were completed, participants were asked to repeat the task of pulling their finger while listening to a tone for four more trials, one trial for each sound condition and for each finger position, with the presentation order randomized across participants. After each trial participants completed the same questionnaire employed in Experiment 1.
Data Availability. The datasets generated during and analysed during the current study are available in the UK Data Service ReShare repository, http://reshare.ukdataservice.ac.uk/852739/.