Abstract
Hand movements are pulled in the direction of motion near their planned endpoints. Is this an automatic response to motion signals near those positions, or do we consider what is moving? To find out, we asked participants to hit a target that moved rightward across a patterned surface when it reached an interception zone that was indicated by a circle. The circle was initially at the center of a square. The square was either filled, occluding the patterned surface (tile), or open, such that the patterned surface was not occluded (frame). The square briefly moved leftward or rightward shortly after the target appeared. Thus, participants were either aiming to hit the target on the surface that moved (the tile) or to hit the target on the patterned surface that did not move. Moving the two types of squares produced very similar local motion signals, but for the tile this could be interpreted as motion of an extended surface, while for the frame it could not. Motion onset of the two types of squares yielded very similar responses. Increasing the size of the square, and thus the eccentricity of the local motion signal, reduced the magnitude of the response. Since this reduction was seen for both types of squares, the surface on which the interception zone was presented was clearly not considered. We conclude that the response is driven by local motion signals near the endpoint of the action without considering whether the local surface is moving.
Similar content being viewed by others
Introduction
We rely on the visual information in our surrounding to successfully interact with the world. The visual scene that is projected onto the retina is organized into coherent regions: the objects and surfaces that constitute our environment. Gestalt psychologists have identified many criteria for segmenting visual scenes into regions, such as similarity, proximity, closure, and connectedness (for reviews, see Peterson & Kimchi, 2013; Rock & Palmer, 1990; Wagemans et al., 2012). Each region of a visual scene could be interpreted as an object or as a surface behind an object (Rubin, 1921). Such interpretation is important for both object recognition and interacting with objects in our environment.
In daily life, we often make visually guided reaching movements to intercept or pick up items. Such movements need to be adjusted when something changes such as the item you are reaching for falling over or you yourself shifting because someone bumped into you. It is well documented that the moving arm quickly follows target displacements (reviewed by Smeets, Oostwoud Wijdenes, & Brenner, 2016). It is also known that the moving arm quickly follows motion onsets in the surrounding area (e.g., Brenner & Smeets, 1997; Gomi, Abekawa, & Nishida, 2006; Saijo, Murakami, Nishida, & Gomi, 2005; Whitney, Westwood, & Goodale, 2003; Zhang, Brenner, Duysens, Verschueren, & Smeets, 2018, Zhang, Brenner, Duysens, Verschueren, & Smeets, 2019), in particular if they occur near the anticipated movement endpoint (e.g., Brenner & Smeets, 2015; Crowe, Smeets, & Brenner, 2021). We propose that this following response is due to shifting the planned movement endpoint in the direction of the motion onset (Crowe et al., 2021). The position of the planned endpoint obviously only changes if the motion onset pertains to the surface at which the movement is supposed to end, so the interpretation of the scene as consisting of surfaces is relevant. We therefore wondered whether the following response is only driven by motion of the surface on which the movement is supposed to end, or whether it is driven directly by any motion signals near the movement endpoint.
To find out whether the response considers whether the local surface is moving, we compared conditions that produce similarly localized motion signals but have different surface interpretations. We compared the responses to two types of squares (frame and tile) of three sizes that were always initially centerd on the interception zone that was indicated by a circle. The frame did not cover a patterned surface, so the movement ended on the patterned surface. In contrast, the tile covered this patterned surface such that the movement ended on the tile. The squares shifted to the left or right at a given moment after the moving target appeared, so the motion signal generated by the shift was restricted to the squares’ (vertical) edges. For the larger squares, the motion signal is farther from the target, so if only local motion information is considered, the magnitude of the response should decrease with size (Crowe et al., 2021). If the response considers whether the local surface is moving, the predictions differ between the frame and tile. For a frame there is no reason to interpret the motion as being near the movement endpoint, especially if the frame is large. In contrast, for a tile the whole area near the movement endpoint belongs to the moving surface, irrespective of the size of the tile. Therefore, if the response considers whether the local surface is moving, one would expect a much smaller response for the frame than for the tile, especially when they are large.
Method
Participants
Twelve participants (11 right-handed; age 27.9 ± 2.3 years; graduate students; convenience sample based on the heuristics justification – Lakens, 2021) volunteered to take part in the experiment. The study was approved by the local ethics committee (Ethics Committee of the Vrije Universiteit Amsterdam) in accordance with the tenets of the Declaration of Helsinki.
Set-up
The experiment was conducted in a normally illuminated room. The stimuli were back-projected at 120 Hz with a resolution of 800 × 600 pixels onto a 1.25 × 1.0-m acrylic rear-projection screen (Techplex 15, Stewart Filmscreen Corporation, Torrance, CA, USA) tilted backward by 30°. Participants stood in front of the screen. An infrared camera (Optotrak 3020, Northern Digital) that was placed at about shoulder height to the left of the screen measured the position of a marker (an infrared light-emitting diode) attached to the nail of the index finger of the participant’s dominant hand at 500 Hz.
In order to synchronize the movement data (i.e., the marker position) with the stimulus presentation, the camera also recorded the position of a second marker attached to the side of the screen. This marker did not move but it stopped emitting infrared light so that its position was registered as “missing” when a flash was presented at the top left corner of the screen (where a light-sensor was placed to detect the flash). We used a simple four-point calibration to relate the position of the fingertip to the projected images, automatically correcting for the fact that the marker was attached to the nail rather than the tip of the finger.
Stimulus and procedure
Participants stood in front of the large screen and were free to move as they wished. The display included four key components that were always visible (see Fig. 1B): a grey background, a layer of 600 randomly positioned 1-cm diameter black discs, a circle indicating the interception zone (6-cm diameter black ring) and a blue square (with sides of either 7, 37, or 67 cm). These components of the display remained static throughout the trial, apart from the blue square that moved at a fixed time on every trial. The interception zone was presented at the horizontal center of the screen 10 cm above the vertical center. The blue square was always initially centered on the interception zone. To give the impression that the interception zone was on the square, the square was a blue tile: the square was filled and occluded the layer of black dots. To give the impression that the interception zone was not on the square, the square was just a frame: only the outline was blue, and even the outline did not occlude the layer of black dots when they overlapped. The width of the frame was 5% of the width of the square. Importantly, in both cases the square’s motion only gives rise to changes in the image itself (i.e., motion signals) at the edges of the square, but for the tile the area at which people are supposed to hit the target is part of the moving surface, whereas for the frame it is not.
At the beginning of each trial, a black starting point (4-cm diameter disc) was presented 30 cm below the interception zone. To start a trial, participants placed the index finger of their dominant hand on the starting point. Between 500 and 700 ms after they did so, the starting point disappeared, and a black target (2-cm diameter disc) appeared 20 cm to the left of the center of the interception zone. This target moved rightward across the screen at 30 cm/s so that it reached the center of the interception zone 667 ms after it appeared. Participants were instructed to tap the target within the interception zone. From 300–400 ms after the target appeared the square moved either rightward or leftward at a constant speed of 20 cm/s (covering 2 cm during the 100 ms of motion). A short epoch of motion was used so that there was no motion at the time of interception, ensuring that the haptic feedback of no motion when participants hit the screen was consistent with the visual information (Cuijpers, Brenner, & Smeets, 2008; Schenk, 2012).
In order to provide participants with feedback on their hitting performance, we detected taps on-line. A tap was detected if the reduction in the distance to the screen between consecutive measurements decreased by more than 1 mm (i.e., a deceleration threshold of 50 m/s2) while the finger was less than 2 cm above the screen. If the position of the fingertip (as determined during calibration) was within the outline of the target, we considered the target to have been hit. If a target was hit it remained at the position at which it was hit for 500 ms. If the position of the fingertip was also within the interception zone, there was a sound indicating that the hit was successful. If a target was missed, it deflected away from the finger at 100 cm/s, also remaining visible for 500 ms unless it left the screen before that.
Design
Two factors were manipulated in a within-subjects design: the type and size of the square. The square was either a frame or a tile (see Fig. 1). The lengths of the edges of the square were 7, 37, or 67 cm. The square moved on every trial. On half of the trials it moved leftward and on the other half of the trials it moved rightward. In total there were 25 leftward and 25 rightward trials for each of the six conditions (two types of square; three sizes) in each block. Participants completed two such blocks of 300 trials each, giving a total of 600 trials, in a single session that took approximately 45 min including a short break between the blocks. All conditions were randomly interleaved.
Data analysis
To evaluate the time-course of the response to the motion of the squares, we first converted the measured lateral positions of the finger into lateral velocities by direct differentiation. This was done for every 2-ms interval for the first 300 ms after the square started to move (300–600 ms after the appearance of the moving target). For each participant, we then averaged the lateral velocity of the finger for every interval. We did so separately for trials in which the square moved leftward and ones in which it moved rightward. We subsequently determined the response by subtracting the average velocity for leftward motion from that for rightward motion. We did this for each condition of the experiment.
Trials with timing issues in the onset or offset of the square’s motion (< 1%) or with missing data (1%) were removed from the analysis. All other trials were included in the analysis, irrespective of performance. After determining the individual responses in each condition, the values were averaged across participants. We present the time-course of the responses to the motion signal as mean values with standard errors across participants. We quantified the response magnitude of the hand for each participant and condition by taking the mean response between 150 ms and 200 ms after the square started moving.
Results
This experiment was not preregistered. All summary level data and the analysis code are available from the Open Science Framework repository at: https://osf.io/4vt59/?view_only=97c2e5cba7f545138695237d0b3d76aa. Participants hit the screen 662 ± 27 ms after the target appeared and hit the target within the interception zone on 70 ± 11% of the 600 trials (mean ± standard deviation across participants’ mean values).
The response of the hand (i.e., the difference in lateral hand velocity after leftward and rightward shifts) was similar for the two kinds of squares, evidenced by the almost perfectly overlapping golden and turquoise curves in Fig. 2. When the size of the square increases, the magnitude of the response decreases. This holds for most individual participants for both kinds of squares (Fig. 3A). The similarity of the response for the two types of squares was present for each combination of size and participant (Fig. 3B). The pattern of results is consistent with the response being driven by any motion near the endpoint of one’s action without considering whether the surface at which the movement ends is moving.
Discussion
People continuously use visual information to adjust their movements in dynamic environments (Brenner & Smeets, 2018). For example, when reaching to a target item their movement is pulled in the direction of motion of the target (Brenner, Smeets, & De Lussanet, 1998; Brouwer, Middelburg, Smeets, & Brenner, 2003; de Lussanet, Smeets, & Brenner, 2001) but also by motion onset in the surrounding area of that item (e.g., Brenner & Smeets, 1997; Gomi et al., 2006; Saijo et al., 2005; Whitney et al., 2003; Zhang et al., 2018, 2019), presumably to stabilize the ongoing movement (Crowe et al., 2021). Since the visual system organizes its input into perceptually coherent objects and surfaces, it makes sense that this response to motion onsets might consider whether the surface at the anticipated movement endpoint is moving, and if not, how close the moving surface is to that endpoint. In this experiment, however, we found that the size of the response to motion onset was the same irrespective of whether the endpoint of one’s action was on a moving tile or a static background. The response was weaker when the motion signal was further away from the movement endpoint. These findings were consistent across the majority of participants. This suggests that the response does not consider whether the surface at which the movement ends is moving, but relies on the motion signal itself.
The similarity between participants’ responses to motion of the tile and of the frame clearly shows that the response did not consider the surface on which the endpoint of one’s action was located. The finding that the response is driven by the onset of any motion in the surrounding area, irrespective of what is moving, is consistent with finding a similar response when obstacles near the target move (Aivar, Brenner, & Smeets, 2008). The fact that in that case the response even occurred when a useful response would be in the opposite direction provides an additional argument against a more elaborate interpretation of the scene. The automaticity of the motion onset response is also consistent with research showing that the response is similar when the reaching task is performed in isolation and when it is performed alongside a second attentionally demanding counting task (De Dieuleveult, Brouwer, Siemonsma, Van Erp, & Brenner, 2018).
In line with other research (e.g., Brenner & Smeets, 2015; Crowe et al., 2021), we find evidence that the response is most sensitive to motion signals close to the endpoint of one’s action. This is in contrast to the conclusions drawn by Abekawa and Gomi (2010), who suggested that the response was not modulated by the location of the motion signal relative to the planned movement endpoint. They found no difference between the hand’s response to motion immediately surrounding the endpoint or separated from the endpoint by a grey mask occluding motion near the endpoint. These authors did not place the edge of the moving pattern further than 17 cm from the anticipated movement endpoint. Since we only found a small reduction in response magnitude when moving from the small- (3.5 cm) to the medium- (18.5 cm) sized squares in our experiment, these results are not too dissimilar from those reported by Abekawa and Gomi (2010). For our larger square size there was a clear decrease in the magnitude of the response. Even for our largest square we did see some response. It remains to be seen how distant the motion signal must be to abolish the response completely.
In this experiment, we find that the response to motion onset in the surrounding area of an ongoing movement does not take into consideration whether it is the surface at which the movement is planned to end that is moving. Instead, it is driven by a fast, automatic mechanism that detects and responds to any motion signal in the vicinity of the endpoint of one’s action. The response is modulated by the location of the motion signal relative to the endpoint of one’s action. It is smaller when the motion signal is further away, but whether the surface on which the movement endpoint is located could be interpreted as moving is irrelevant.
Data availability
The dataset generated and analyzed during the current study are available on the Open Science Framework Repository (https://osf.io/4vt59/?view_only=97c2e5cba7f545138695237d0b3d76aa).
Code availability
The code used to analyze the data sets is available on the Open Science Framework Repository (https://osf.io/4vt59/?view_only=97c2e5cba7f545138695237d0b3d76aa).
References
Abekawa, N., & Gomi, H. (2010). Spatial Coincidence of Intentional Actions Modulates an Implicit Visuomotor Control. Journal of Neurophysiology, 103, 2717–2727. https://doi.org/10.1152/jn.91133.2008
Aivar, M. P., Brenner, E., & Smeets, J. B. J. (2008). Avoiding moving obstacles. Experimental Brain Research, 190, 251–264. https://doi.org/10.1007/s00221-008-1475-9
Brenner, E., & Smeets, J. B. J. (1997). Fast responses of the human hand to changes in target position. Journal of Motor Behaviour, 29, 297–310. https://doi.org/10.1080/00222899709600017
Brenner, E., & Smeets, J. B. J. (2015). How moving backgrounds influence interception. PLoS One, 10, e0119903. https://doi.org/10.1371/journal.pone.0119903
Brenner, E., & Smeets, J. B. (2018). Continuously updating one’s predictions underlies successful interception. Journal of Neurophysiology, 120, 3257–3274. https://doi.org/10.1152/jn.00517.2018
Brenner, E., Smeets, J. B. J., & De Lussanet, M. H. (1998). Hitting moving targets. Continuous control of the acceleration of the hand on the basis of the target’s velocity. Experimental Brain Research, 122, 467–474. https://doi.org/10.1007/s002210050535
Brouwer, A. M., Middelburg, T., Smeets, J. B. J., & Brenner, E. (2003). Hitting moving targets: a dissociation between the use of the target’s speed and direction of motion. Experimental Brain Research, 152, 368–375. https://doi.org/10.1007/s00221-003-1556-8
Crowe, E. M., Smeets, J. B. J., & Brenner, E. (2021). The response to background motion: characteristics of a movement stabilization mechanism. Journal of Vision, 21, 3. https://doi.org/10.1167/jov.21.11.3
Cuijpers, R. H., Brenner, E., & Smeets, J. B. J. (2008). Consistent haptic feedback is required but it is not enough for natural reaching to virtual cylinders. Human Movement Science, 27, 857–872. https://doi.org/10.1016/j.humov.2008.07.003
de Dieuleveult, A. L., Brouwer, A. M., Siemonsma, P. C., Van Erp, J. B. F., & Brenner. (2018). Aging and sensitivity to illusory target motion with or without secondary tasks. Multisensory Research, 31, 227–249. https://doi.org/10.1163/22134808-00002596
de Lussanet, M. H., Smeets, J. B. J., & Brenner, E. (2001). The effect of expectations on hitting moving targets: influence of the preceding target's speed. Experimental Brain Research, 137, 246–248. https://doi.org/10.1007/s002210000607
Gomi, H., Abekawa, N., & Nishida, S. (2006). Spatiotemporal tuning of rapid interactions between visual-motion analysis and reaching movement. Journal of Neuroscience, 26, 5301–5308. https://doi.org/10.1523/JNEUROSCI.0340-06.2006
Lakens, D. (2021). Sample size justification. PsyArXiv.https://doi.org/10.31234/osf.io/9d3yf
Peterson, M. A., & Kimchi, R. (2013). Perceptual organization. In D. Reisberg (Ed.), The Oxford handbook of cognitive psychology (pp. 9–31). Oxford University Press.
Rock, I., & Palmer, S. (1990). The legacy of Gestalt psychology. Scientific American, 263, 84–91.
Rubin, E. (1921). Visuell wahrgenommene Figuren. Gyldendalske Boghandel.
Saijo, N., Murakami, I., Nishida, S., & Gomi, H. (2005). Large-field visual motion directly induces an involuntary rapid manual following response. Journal of Neuroscience, 14, 3208–3224. https://doi.org/10.1523/JNEUROSCI.4143-04.2005
Schenk, T. (2012). Response to Milner et al.: Grasping uses vision and haptic feedback. Trends in Cognitive Sciences, 16, 258–259. https://doi.org/10.1016/j.tics.2012.03.006
Smeets, J. B. J., Oostwoud Wijdenes, L., & Brenner, E. (2016). Movement adjustments have short latencies because there is no need to detect anything. Motor Control, 20, 137–148. https://doi.org/10.1123/mc.2014-0064
Wagemans, J., Elder, J. H., Kubovy, M., Palmer, S. E., Peterson, M. A., Singh, M., & von der Heydt, R. (2012). A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure–ground organization. Psychological Bulletin, 138, 1172–1217. https://doi.org/10.1037/a0029333
Whitney, D., Westwood, D. A., & Goodale, M. A. (2003). The influence of visual motion on fast reaching movements to a stationary object. Nature, 423, 869–873. https://doi.org/10.1038/nature01693
Zhang, Y., Brenner, E., Duysens, J., Verschueren, S., & Smeets, J. B. J. (2018). Postural responses to target jumps and background motion in a fast pointing task. Experimental Brain Research, 236, 1573–1581. https://doi.org/10.1007/s00221-018-5222-6
Zhang, Y., Brenner, E., Duysens, J., Verschueren, S., & Smeets, J. B. J. (2019). Is the manual following response an attempt to compensate for inferred self-motion? Experimental Brain Research, 237, 2549–2558. https://doi.org/10.1007/s00221-019-05607-x
Funding
This work was supported by the Netherlands Organisation for Scientific Research (NWO) under project number 464.18.111 awarded to Eli Brenner.
Author information
Authors and Affiliations
Contributions
EC and EB were responsible for the conception and design of the study. EC was responsible for data collection and analysis. All authors were involved in the interpretation of the data and writing of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval
Ethical approval was obtained from the Ethics Committee of the Vrije Universiteit Amsterdam.
Consent to participate
Informed consent was obtained from all participants included in the study.
Consent to publication
Not applicable.
Conflicts of interest
The authors declare that they have no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Statement of Significance
In daily life, we rely on visual information to guide our goal-directed actions. These actions can be adjusted within 100 ms to account for changes in the environment. Unravelling what contributes to the adjustments of our movements is essential in understanding how the brain organises the incoming retinal input. Since the visual system automatically organizes its input into the objects and surfaces that constitute our environment, we asked whether visually guided movements also consider these underlying scene configurations. Participants’ movements were similar irrespective of the object-surface configuration, showing that a fast and automatic system underlies the online control of visually guided action.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Crowe, E.M., Smeets, J.B.J. & Brenner, E. Hand movements respond to any motion near the endpoint. Atten Percept Psychophys 84, 1820–1825 (2022). https://doi.org/10.3758/s13414-022-02471-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13414-022-02471-w